Published: Aug 8, 2020 by Greg Woodbury
Traditional Data Warehousing for buy-side managers has evolved. Gone are the days when firms are solely looking for a centralized data store for their most important data assets. These data stores solve the single source of truth problem, but they fall woefully short when managers need to gain advanced insights to help drive their decision making. In our experience, most resort to manually pulling data into excel spreadsheets to solve problems that should be handled by an Enterprise Data Management Solution (EDMS).
What is Enterprise Data Management?
EDM is a holistic approach to a firm’s data. It includes the ingestion, processing, cleaning, analytics and presentation of data. It seeks to integrate data from many different sources inside and outside of your firm, and it does so through a framework that improves the integrity and trustworthiness of the data. It also helps firms coordinate and audit the handling of data between systems and business processes.
Each of these areas of concern could have their own dedicated book written about them. We know, we’ve written quite a few articles on the topic. Let’s focus on the high level key aspects that you should demand from your EDMS.
Data Integration
When speaking about data integration, most people think of getting data into the centralized Data Warehouse, but this is only the tip of the iceberg. Yes, your system should allow you to seamlessly integrate with all of your 3rd parties including your Market Data Provider, Fund Admin, Prime Brokers, Accounting, Trade Execution, and Order Management systems, but it should go further. It should be the central location where business processes are tied together and triggered.
Do you need to alert your trade desk when there has been a large market move? Your EDMS should handle that.
Do you need to alert your operations department when data from your brokers, accounting system and fund admin don’t tie? Your EDMS should handle that.
You should be able to tell your EDMS about how your firm operates so it can tie related processes together. This way nothing is ever dropped and everything is always tracked and recorded. This also opens the door to greater automation opportunities and eliminates the need for manual hand-offs.
Data Quality
Data is messy. It rarely arrives in the shape you expect, nor does it contain a complete picture by itself. In traditional Data Management, there are often whole teams dedicated to the extraction, transformation and loading of data. They implement rules to scrub and aggregate the data to conform with the way you think about the world. Even with all of the effort to create and maintain these rules, there are invariably instances which arise that cause unexpected errors in the system. These errors can lead to costly downtime in the system while the source of the error is investigated and the system patched to handle the new unexpected data. Even if you are using a Data Warehouse solution which allows for a higher level configuration, it still requires a team of specialists to understand how to implement the configuration and maintain it over time.
There is a better way. Your EDMS should have intimate knowledge about your 3rd party data sources and you should be able to integrate with them, simply by telling your EDMS which sources you care about. No complex configuration, no rules maintenance.
Timeliness
Often we find that most existing Data Warehouse solutions are slow. They are built with the assumption that data only needs to be ingested at a few key times throughout the day. These times are largely driven by 3rd party SLAs for the delivery of data or around internal business processes. Generally speaking this is ok for a small number of data sources, however, as your firm grows and wants to bring on more data, this “batch processing” can lead to scheduling nightmares and slow loading times.
What if data from external sources arrives late?
What if bad data arrives from a 3rd party and needs to be resent?
How do you know when your data is in a good state for downstream processes?
Rather than relying on a schedule, your EDMS should be “alive” and working for you 24/7. No matter when data arrives, it should be immediately ingested and available for appropriate consumption.
Accessibility/Auditability
A major concern that is often overlooked or poorly implemented is the security of your data. Today, hackers are sophisticated and are continuously trying to break into your operation. There are far too many examples of companies who did not take the time to consider the security of their data until it was too late. Not only does this ruin trust between you and your clients, but it is costly and time consuming to remediate.
We know that cumbersome security protocols can make it difficult for your team to access the data that they need to complete their jobs, but this needs to be balanced with making sure that your data stays safe. Your system should make sure that you are applying best practices as well as track and audit all activity. This way you are reducing your vulnerability and increasing the likelihood of detecting anomalous behavior. In the event of a breach, you will be armed with the information to determine what happened and how.
Analytics
The raw data that comes in from your 3rd party vendors is not enough to run your business. You need to apply your own analytics on top of that data to derive insights and drive your business forward. As mentioned above, this is typically achieved by manually pulling the raw data into Excel spreadsheets and running your custom calculations. The problem with that is that it is hard to organize and does not scale. If you need to apply that same calculation in multiple different places across your organization you are opening yourself up to the possibility that each implementation might be slightly different. This will in turn increase the time it takes to reconcile the different calculations and will decrease the trust of the data across the firm.
Important metrics and calculations should be baked into your EDMS. Centralizing the logic for these analytics ensures that you get consistent data every time you access it. No guesswork, no complex reconciliations.
Data Communication
Data doesn’t do anyone any good unless you can use it to tell a story. This is why we believe that an EDMS is the right place to source your data for internal and external communication. Here are a few options to consider. Should you build all of your reporting directly into your system or can you leverage leading reporting and business intelligence tools? How about a mix of both? Answers to these questions are ones that will be different for every firm. Your EDMS should not lock you into a single path for communications.
Closing Thoughts
Strategies of the past are no longer sufficient to tackle today’s business and data problems. With a modern and well architected Enterprise Data Management solution you can make sure that your firm stays agile in the face of ever changing demands for data and data consistency.
If you would like to find out more about how Red Arris’ team of industry leaders is driving this change, please reach out. We love chatting about financial data.
