Anaconda team is giving away a free O'Reilly ebook Breaking Data Science Open:
See below some extracts from its preface:
Data science has captured the public’s attention over the past few
years as perhaps the hottest and most lucrative technology field.
...
The leading edge of this tsunami is a combination of innovative business and technology trends that promise a more intelligent
future based on the pairing of open source software and crossorganizational
collaboration called Open Data Science. Open Data
Science is a movement that makes the open source tools of data science—data, analytics, and computation—work together as a connected ecosystem.
...
This report discusses the evolution of data science and the technologies behind Open Data Science, including data science collaboration, self-service data science, and data science deployment. Because
Open Data Science is composed of these many moving pieces, we’ll
discuss strategies and tools for making the technologies and people
work together to realize their full potential.
Showing posts with label Open Source. Show all posts
Showing posts with label Open Source. Show all posts
Friday, 10 March 2017
Tuesday, 1 September 2015
Brief Q&As About Solution Architecture
1. What would be the key artefacts that need to be delivered when defining a solution architecture?
Key artefacts present
different views on solution design where each view is targeting specific
user groups. For example:
- Business architecture artefacts address business users, planners and business stakeholders.
- Data architecture artefacts address database and system administrators.
- Application architecture artefacts provide guidelines and instructions for software development and test teams.
- Technology architecture artefacts are essential for infrastructure administrators, development team and system managers.
- Security architecture artefacts address the development and test teams, system administrators, security auditors, business users and managers.
Solution architecture artefacts also
depend on methodology used by an organisation that the solution is designed for.
Popular methodologies include but not limited to TOGAF, Zachman, RUP, CMMI,
FEAF. Examples of key artefacts could
be:
- TOGAF:
- Catalogues (role catalogue, product catalogue, data entity catalogue, etc)
- Matrices (business interaction matrix, actor/role matrix, system data matrix, etc)
- Core and extension diagrams (functional decomposition diagram, use case diagram, process flow diagram, event diagram, product lifecycle diagram, data security diagram, etc)
- Zachman:
- Practically sufficient subset of Lists, Models, Diagrams, Specifications and Details documents that are based on 30 views of Zachman framework matrix – Why/How/What/Who/Where/When by Contextual/Conceptual/Logical/Physical/Detailed.
Sometimes a final Architecture Document
could combine major views and present the solution to most user groups. From my
experience I found that in some cases that require relatively urgent delivery,
a properly built Business Requirements Document could become a part of the
final Architecture Document. I published a short manual for such BRD on my blog
at http://bananaqualitytester.blogspot.co.uk/2013/01/guidelines-for-business-requirements.html.
2. What would be the role of Solution Architect during the project lifecycle?
Solution Architect’s activity during the
project lifecycle should be organically coupled with business stakeholders’
decision making, development team/s delivery progress and the end users
expectations to ensure:
- Target architecture would accommodate any possible evolution of the business stakeholders’ vision of the future system.
- Development teams would follow the architectural guidelines without sacrificing quality, robustness and effectiveness of the future system.
- The end users have an early access to the system-in-development, play with it, test it and provide feedback to architecture and development teams.
It always a good idea to keep architecture
artefacts such as definitions, models, diagrams, specifications, etc updated
during the project to make sure they could be re-used in the future by the
business users, help desk and system support staff.
3. How would a Solution
Architecture work in the context of Waterfall and Agile methodologies?
Once all business requirements are
clarified and understood, a typical search for a solution may include such
activities as reviewing competitive technologies, evaluating open and
commercial off-the-shelf products, building and testing prototypes, etc. In
some organisations it’s called Research and Development (R&D) stage. The
output of R&D stage is a solution design that is (or at least very close to)
target architecture that would be eventually defined in such documents as
Architecture Document mentioned above.
Waterfall methodology is acceptable in
cases when business decision about future system is final and a chance for any
changes is very low. In this case R&D stage is paramount for the whole
project as the solution architecture artefacts would be translated into project
management timelines, delivery milestones, QA test case scenarios and system
releases. Given the importance of R&D, it would make sense to spend more
time on avoiding any ambiguity during reviewing business, functional and
non-functional requirements, looking for possible issues with off-the-shelf
products and stress-testing prototypes to ensure a delivery of a flawless
solution.
Agile methodology doesn’t require
absolutely complete set of requirements, giving it a chance to evolve during
the system development. From high-level perspective, an agile project could be represented
as follows:
The straight lines coming from the centre
(epicentre) are individual use cases, circle lines are delivery milestones and
the red spiral line is actual system development that ‘covers’ use cases (and
milestones) more and more with each iteration until all of them are done (or
delivered). In this case the core system functionality (area around the epi-centre)
is paramount and requires clear prioritisation by the solution architect.
Because of project agile nature, it is expected that solution architecture
artefacts would evolve during the project and therefore should be continuously
updated by the architect and those changes propagated through all teams
involved into the product delivery.
4. What process steps would be expected between
the capture of requirements and start of the coding?
These steps may depend on architecture
methodology used in the organisation (TOGAF, RUP, Zachman, etc) as well as the
type of given project (waterfall or agile). They may include:
- Business, functional and non-functional requirements review and clarification.
- Business requirements update (see Guideline for Business Requirements mentioned above) and their verification with business users. These steps would help to build with business users unambiguous project vocabulary, learn more about their expectations, identify key people in their team and establish working relationships with them.
- R&D stage – please, see (3) for more details.
- Initial solution design.
- Verification of acceptability of proposed solution for existing or target infrastructure.
- Cross-check of proposed solution design with budget requirements.
- Validation of proposed solution design against system security constraints.
- Validation of proposed solution design against internal and/or external rules and legislation.
- Approval of solution design with key stakeholders.
- Final (or semi-final in case of agile) solution design.
- Creation of solution architecture artefacts - please, see (1) for more details.
- Definition of development environment. This may include continuous integration and automated testing, knowledge management system, issue tracking system, network topology (in case of delivering a Cloud-based solution), etc – this is a range of approaches that would allow to keep development process transparent to the architecture team as well as to other interested parties involved in the project.
- Requirements for system test coverage.
Labels:
Agile,
architecture,
artefact,
AWS,
Cloud,
design,
Document,
Domain,
FEAG,
Mobile,
Open Source,
RUP,
solution,
Specification,
TOGAF,
Use Case,
Waterfall,
Workflow,
Zachman
Location:
London, UK
Friday, 8 February 2013
Microsoft Office on Linux in 2014?
What if it's true and after all we'll get a full version of Microsoft Office available on Linux platform? Let's see who are the main players in this space:
2. LibreOffice (free)
3. SoftMaker Office ($79.95)
4. IBM Lotus Symphony (free, based on OpenOffice)

5. GNOME Office (free)

6. KOffice (free)

7. StarOffice (free)

8. Calligra (free)

- Google Docs Office Suite (free)
- Microsoft Office 365 ($4-20 per user/month)
- CollateBox ($9.99-129.99/month for unlimited number of users)
It's pretty clear that Microsoft won't be alone out there. It would be quite interesting to see how competition would change the lists above.
DESKTOP
1. OpenOffice.org (free)
3. SoftMaker Office ($79.95)
4. IBM Lotus Symphony (free, based on OpenOffice)

5. GNOME Office (free)
6. KOffice (free)

7. StarOffice (free)

8. Calligra (free)
9. Yozo Office ($59)
10. Breadbox ($99)
ONLINE
- ThinkFree Online Office ($49)- Google Docs Office Suite (free)
- Microsoft Office 365 ($4-20 per user/month)
- CollateBox ($9.99-129.99/month for unlimited number of users)
It's pretty clear that Microsoft won't be alone out there. It would be quite interesting to see how competition would change the lists above.
Tuesday, 29 January 2013
Open Source Financial Trading Software
Some time in early 2012 an open source project Lodestone Foundation was backed by Deutsche Bank. In September, 2012 FT let it know to ones who missed the news earlier.
The initiative is quite interesting and it split finance professionals in two camps - ones who believe that this is the way to go and their counterparts who think it would never happen.
I've looked for any existing and more or less active projects in this space. This is what I found out so far:
The OpenGamma Platform provides both buy-side and sell-side firms a rich set of features, including trade data management, live risk alerts, and sophisticated, dynamic Excel integration.
QuantLib project is aimed at providing a comprehensive software framework for quantitative finance. QuantLib is a free/open-source library for modeling, trading, and risk management in real-life.
Possibly this is the most popular project at the moment, there are signs that it's quietly used by many quant developers in their day-to-day work.
Open Heartbeat is a lightweight data distribution system designed primarily for collecting and distributing rapidly changing price quotes for stocks, securities or other distributable data sets.
The core component is a reusable library which manages all communication, synchronization caching etc., which can be linked into applications that create or, more commonly, consume data.
Primarily aimed at the financial sector, as a way to distribute rapidly changing stock quotes etc., this system can be used for any application that needs to distribute volatile data sets reliably and efficiently.
This suite includes a sample feed handler for Yahoo financial data. Use of the data provided by this service is subject to the terms and conditions of the Yahoo real time stock quote service.
The library offers very efficient operation (~10,000 updates per second), redundant paths and automatic error recovery and reporting.
EclipseTrader is an Eclipse Rich Client Platform (RCP) application focused on the building of an online stock trading system, featuring shares pricing watch, intraday and history charts with technical analysis indicators, level II/market depth view, news watching, and integrated trading.
The standard Eclipse RCP plug-ins architecture allows third-party vendors to extend the functionality of the program to include custom indicators, views or access to subscription-based data feeds and order entry.
The Financial Information eXchange (FIX) protocol is a messaging standard developed specifically for the real-time electronic exchange of securities transactions. FIX is a public-domain specification owned and maintained by FIX Protocol, Ltd (FPL).
QuickFIX/J is a fully featured messaging engine for the FIX protocol. It is a 100% Java open source implementation of the popular C++ QuickFIX engine.
Premium Markets is an automated financial technical analysis system.
It implements a graphical environment for monitoring financial technical analysis major indicators and for portfolio management.
In its advanced packaging (not provided under open-source license) it also includes :
ojAlgo is Open Source Java code that has to do with mathematics, linear algebra and optimization Its feature set make it particularly suitable for use within the financial domain.
EWaveTrade is a tool to aid in the process of analyzing market price data using the Elliot Wave Theory, particularly applying the concepts and principles developed by Glenn Neely. The book "Mastering Elliott Wave" by Glen Neely is recommended for both potential developers and users of this software.
QChartist is a free charting software designed to do technical analysis from any data. The program is written in Basic language.
Pure Java lib that provides:
ActiveQuant is an entire framework to backtest and run automated trading systems. This project is an open source project, but is not a free software, it is provided free of charge to academics and hobbyists.
Akutan is an open source finance project working in the asset allocation, portfolio analysis arena of financial informatics. For the most part it stays away from instrument valuation and market models given that there are several other projects dedicated to that space, e.g. quantlib.
Net Positions allows to receive current and historical data about net positions(*) and opened on the futures market (info is taken from 'Commitments of Traders' weekly report).
The initiative is quite interesting and it split finance professionals in two camps - ones who believe that this is the way to go and their counterparts who think it would never happen.
I've looked for any existing and more or less active projects in this space. This is what I found out so far:
1. OpenGamma (Java, R, .Net and REST APIs, Tools for MS Excel)
The OpenGamma Platform provides both buy-side and sell-side firms a rich set of features, including trade data management, live risk alerts, and sophisticated, dynamic Excel integration.
2. QuantLib (C++)
QuantLib project is aimed at providing a comprehensive software framework for quantitative finance. QuantLib is a free/open-source library for modeling, trading, and risk management in real-life.
Possibly this is the most popular project at the moment, there are signs that it's quietly used by many quant developers in their day-to-day work.
3. Open Heartbeat (probably C/C++)
Open Heartbeat is a lightweight data distribution system designed primarily for collecting and distributing rapidly changing price quotes for stocks, securities or other distributable data sets.
The core component is a reusable library which manages all communication, synchronization caching etc., which can be linked into applications that create or, more commonly, consume data.
Primarily aimed at the financial sector, as a way to distribute rapidly changing stock quotes etc., this system can be used for any application that needs to distribute volatile data sets reliably and efficiently.
This suite includes a sample feed handler for Yahoo financial data. Use of the data provided by this service is subject to the terms and conditions of the Yahoo real time stock quote service.
The library offers very efficient operation (~10,000 updates per second), redundant paths and automatic error recovery and reporting.
4. Eclipse Trader (Java)
EclipseTrader is an Eclipse Rich Client Platform (RCP) application focused on the building of an online stock trading system, featuring shares pricing watch, intraday and history charts with technical analysis indicators, level II/market depth view, news watching, and integrated trading.
The standard Eclipse RCP plug-ins architecture allows third-party vendors to extend the functionality of the program to include custom indicators, views or access to subscription-based data feeds and order entry.
5. QuickFX/J (Java)
The Financial Information eXchange (FIX) protocol is a messaging standard developed specifically for the real-time electronic exchange of securities transactions. FIX is a public-domain specification owned and maintained by FIX Protocol, Ltd (FPL).
QuickFIX/J is a fully featured messaging engine for the FIX protocol. It is a 100% Java open source implementation of the popular C++ QuickFIX engine.
6. Premium Markets (Java)
Premium Markets is an automated financial technical analysis system.
It implements a graphical environment for monitoring financial technical analysis major indicators and for portfolio management.
In its advanced packaging (not provided under open-source license) it also includes :
- Screening of financial web sites to pickup the best market shares,
- Forecast of share prices trend changes on the basis of financial technical analysis, (with a rate of around 70% of forecasts being successful observed while back testing over DJI, FTSE, DAX and SBF),
- Back testing and Email sending on buy and sell alerts triggered while scanning markets and user defined portfolios.
7. ojAlgo (Java)
ojAlgo is Open Source Java code that has to do with mathematics, linear algebra and optimization Its feature set make it particularly suitable for use within the financial domain.
8. EWaveTrade (C++, MFC, Visual C++ Compiler)
EWaveTrade is a tool to aid in the process of analyzing market price data using the Elliot Wave Theory, particularly applying the concepts and principles developed by Glenn Neely. The book "Mastering Elliott Wave" by Glen Neely is recommended for both potential developers and users of this software.
9. QChartist (Qt, Basic)
QChartist is a free charting software designed to do technical analysis from any data. The program is written in Basic language.
10. jFin (Java)
Pure Java lib that provides:
- Date adjustment
- Schedule generation
- Day count fraction calculation
11. ActiveQuant (Java, ActiveMQ, HSQL)
ActiveQuant is an entire framework to backtest and run automated trading systems. This project is an open source project, but is not a free software, it is provided free of charge to academics and hobbyists.
12. Akutan (Java)
Akutan is an open source finance project working in the asset allocation, portfolio analysis arena of financial informatics. For the most part it stays away from instrument valuation and market models given that there are several other projects dedicated to that space, e.g. quantlib.
13. Net Positions
Net Positions allows to receive current and historical data about net positions(*) and opened on the futures market (info is taken from 'Commitments of Traders' weekly report).
(*) net position is the difference between total open long and open shot positions.
CreditAnalytics is a full-featured financial fixed income credit analytics, trading, and risk library, developed with a special focus towards the needs of the credit products community. In particular, CreditAnalytics provides analytics to value liquid products (CDS, CDX, CDO, and bonds of all types and variants) and standard index/custom products (single credit forwards and options, and portfolio credit forwards, options, tranches, and other structures).
FpML® (Financial products Markup Language) is the open source XML standard for electronic dealing and processing of OTC derivatives. It establishes the industry protocol for sharing information on, and dealing in, financial derivatives and structured products.
The standard is developed under the auspices of ISDA, using the ISDA derivatives documentation as the basis. As a true open standard, the standards work is available to all at no cost and open to contribution from all. There is no membership requirement.
The standard evolution and development is overseen and managed by the FpML Standards committee, following W3C rules of operations guidelines.
The objective of this project is to create a library of code for pricing financial derivatives products using CUDA to achieve GPU programming.
14. Credit Analytics
CreditAnalytics is a full-featured financial fixed income credit analytics, trading, and risk library, developed with a special focus towards the needs of the credit products community. In particular, CreditAnalytics provides analytics to value liquid products (CDS, CDX, CDO, and bonds of all types and variants) and standard index/custom products (single credit forwards and options, and portfolio credit forwards, options, tranches, and other structures).
15. FpML
FpML® (Financial products Markup Language) is the open source XML standard for electronic dealing and processing of OTC derivatives. It establishes the industry protocol for sharing information on, and dealing in, financial derivatives and structured products.
The standard is developed under the auspices of ISDA, using the ISDA derivatives documentation as the basis. As a true open standard, the standards work is available to all at no cost and open to contribution from all. There is no membership requirement.
The standard evolution and development is overseen and managed by the FpML Standards committee, following W3C rules of operations guidelines.
16. Kooderive
The objective of this project is to create a library of code for pricing financial derivatives products using CUDA to achieve GPU programming.
Subscribe to:
Posts (Atom)
Online Encyclopedia of Statistical Science (Free)
Please, click on the chart below to go to the source:
-
Thanks to an excellent Java Concept of the Day , this is a brief description of main interfaces and classes of Java Collection Framework. H...
-
Some time in early 2012 an open source project Lodestone Foundation was backed by Deutsche Bank. In September, 2012 FT let it know to ones...
-
1. Notation: $ m $ - number of training examples. $ n = \vert x^{(i)} \vert $ - number of features. $ x^{(i)} $ - column vector of all...




