by Jane McConnell
OSDU Management Committee Member / O&G Data Architect, Teradata
“If you want to change your life you have to raise your standards.” Tony Robbins
It’s not often you will catch me quoting a motivational speaker – but this is the quote that sprung to mind for today’s blog post. I’ve heard a bit of chatter around what it means to be OSDU Certified, so I think we should talk about the OSDU Technical Standard.
Before I go into the specifics of the OSDU Technical Standard version 1.0, I’d like to remind you all about our philosophy behind the continuous development of the OSDU Data Platform, and how that relates to the way that we will “raise our standards” as we move forward.
Aligned Autonomy
Maybe Spotify said it best in this short video clip.
There are a lot of people involved in the OSDU Forum, and we want to make use of the best minds to find the best ways to meet our goals. If we stay aligned in our high-level goals, we can allow a higher degree of autonomy in individual teams as they find the best solutions to the different areas they are working on. This will let us move faster and, I believe, build a better overall solution.
You can read more about the high level business context and our approach for designing and implementing the OSDU data platform in the OSDU System Concept document (yes, the one that I spoke about in my last blog post Can you see what it is yet?). We now have a complete draft that is worth reading – and you can find a copy here (Member Access required).
Intentional Architecture, Emergent Design
We are creating something new with the OSDU Forum and the Data Platform. If we set a detailed design up front, we limit our thinking to what we already know, and we limit our ability to adjust and learn as we progress. To deliver an IT solution against emerging business requirements in an ever-changing IT environment, we need to allow for emergent design. But as that video clip explained, agility without alignment can easily lead to chaos. Our ability to maintain alignment on the high-level architecture will be key to our success.
We are busy working on an updated Reference Architecture document – and this will be an important asset in achieving and maintaining the needed alignment here.
Standards Follow Design
Standards are an essential part of achieving the goals of the OSDU Forum. We have no doubts as to the importance of standards, and the role that The Open Group plays. And for us, these will mainly be API standards for operating and integrating with the OSDU Data Platform. But because of the above, we are doing things a little bit differently in this space to traditional standards organisations because we are taking learnings from approaches like Lean Startup.
Our approach is to develop working code first, release this code out to the wider community to test its viability, and once it has passed this real-world test, we will lock down the implementation and add these new services and APIs to the OSDU Technical Standard. Design, develop, test first, and standards after.
Now that we have got that out the way, let’s come back to the details for the Mercury release.
OSDU Technical Standard version 1.0
The OSDU Technical Standard – the thing which is considered “normative” and that companies can be certified against – will for version 1.0 of the OSDU Technical Standard cover Platform Provider Certification, allowing us to certify the behaviour of Platform provider implementations of (most of) the Core Services, also known as the Generic APIs.
Using this high-level picture of the OSDU Data Platform, what we will be making a Technical Standard and certifying is the Generic APIs, which tests their implementation down through the common code, through the Service Provider Interface and through to the specific Platform Provider Implementation. The conformance tests are written as black-box tests on the generic APIs, ie the functionality of these APIs must be exactly as documented in the test cases.
For this initial version of the Technical Standard, there will not be any certification of:
- applications that run on the platform
- the DDMS components that will be delivered with/soon after the March 24 date (seismic and wellbore)
- external data sources that will interface with the platform
- the workflow service, or associated ingestion and enrichment frameworks
This is on purpose and by design. We have agreed that our approach with the OSDU Data Platform is first to develop working code, then to test it for its usability in the Real World, and after that – once we know the new things are useful and worth keeping – we will extend the standard to include them. The core services are considered to be already tested in the Real World as they have been ported from Schlumberger’s Data EcoSystem.
The current draft of the Technical Standard has the following detail (this is still subject to change).
The Generic APIs that are currently mandatory for certification are:
Schema, Storage, Indexer, Search, Legal, Delivery, Notification, Registration, Entitlements, File, Unit System, CRS Conversion
The following services are not currently mandatory for certification, but are expected to form part of a future standard:
Partition, CRS Catalog, Workflow, Ingestion, Enrichment
The Technical Standard explains our current thinking for Application Certification & External Data Provider Certification as part of a future standard, but this is informative only at this time. Same for the DDMS components.
Which means – if anyone states their application is certified for OSDU, then they are wrong by definition, because there is no certification for applications at this time! Feel free to remind them of this – it will help minimise confusion. Now, their application may be written in accordance with the OSDU architecture guidelines – which is a Very Good Thing and worth talking about – but applications are not being certified at this point. Application Providers can state that they align with the OSDU Technical Standard. We are only certifying the Platform Providers in this first release.
It also means there is nothing in the standard that says that an OSDU Data Platform Provider cannot provide additional services as well as the generic APIs – but the OSDU generic APIs must exist and their behaviour must conform to the standard in order for their platform to pass certification. These additional services could be contributed back to OSDU common code at some point, and may later become part of a future standard – but if they don’t go that route then these extra services are not OSDU services, they are implementation-specific services.
A snapshot of the current draft of the OSDU Technical Standard can be found here (Member Access required).
And if you’d like to get involved, there’s a Slack channel 1_5_standards_documents dedicated to this work. (Member Access required)
Here’s to continuously raising our standards, and to continuously improving our (data management) life!
Jane McConnell Bio
Coming from an IT background, Jane fell into Upstream Oil & Gas with its specific data management needs back in 2000, and has been consulting, developing product, and implementing solutions in this area since. She has done time at both Landmark and Schlumberger – in R&D, product management, consulting, and pre-sales – and in one role or another, she has influenced subsurface data management projects for most major oil companies across Europe. She chaired the Education Committee for the European oil industry data management group ECIM, and regularly presents at Oil Industry data management events. Along the way she’s learned enough about geophysics, geology, and the world of hydrocarbon exploration and production to be dangerous.
For the last 8 years has been working in Teradata, a big data and analytics company, in their Oil and Gas Practice. She is also serving on the OSDU Management Committee.
Jane is Scottish and has a stereotypical love of whisky (especially Islay malts). She knits and reads almost to obsession.
You can contact her at jane.mcconnell@teradata.com, on the OSDU Slack channels, or connect on LinkedIn.
