Introduction to Structured Data
Structured data refers to a data model and format that organizes information in a predictable, preset manner that is amenable to machine reading and processing. The two main types of structured data are tabular data and interconnected data. Tabular data organizes information into rows and columns, like a simple spreadsheet or database table. Interconnected data uses a graph-like structure of nodes and edges to connect related pieces of data in a web-like manner.
Some key aspects of structured data include:
Schemas and metadata: Structured data relies on predefined schemas or metadata to describe the type, format, and meaning of each piece of data. This allows machines to interpret the data automatically.
Standardized formats: Common formats for structured data include CSV, JSON, XML, SQL databases, RDF triples/graphs. These prescribe both the data structure and syntax.
Relationships: Structured data explicitly captures relationships between different data elements through column headers, unique identifiers, foreign keys, node properties, etc. This relational structure mirrors the real-world relationships between entities.
Queryable: The rigidity of structured data models enable automated querying, filtering, aggregation and analysis over large datasets through SQL, SPARQL etc. Flexible schemas permit ad-hoc queries too.
Machine-readable: With a predefined structure and standard syntax, structured data is readily processable by programs, algorithms and artificial intelligence without extensive human intervention.
Some key uses of structured data include powering databases, data warehouses, knowledge graphs, interactive applications, predictive analytics, search engines etc. Well-structured data is foundational for advanced data-driven technologies.
Implementing Structured Data on the Web
The web initially grew organically with unstructured HTML documents linked together. As websites evolved into complex applications, there was a need to expose structured data programmatically for experiences like search engine optimization, interactive features, recommendation engines etc.
This need gave rise to various structured data formats for labeling web pages with machine-readable metadata. Some prominent ones are:
Microdata: Uses HTML5 ́s itemscope attribute to annotate pages with name-value pairs for properties, types etc inside
Microformats: Encodes semantics as classes in HTML to label contacts, events and other entities. Predecessor to Microdata.
RDFa: Embeds RDF triples as attributes directly in HTML, extending the information on web pages with structured, semantic metadata.
JSON-LD: A JSON-based format to serialize Linked Data. Embedded in HTML script tags or served as separate .jsonld files.
Schema.org: A collaborative vocabulary for structured data across industry segments like products, reviews, people etc. Defines common entities and properties.
Implementing any of these formats on web pages helps search engines, browsers and bots understand the structure and meaning of page content. It also enables richer features like knowledge panel, site links extraction, recipe/review rich results etc in search. Structured data is an essential element of site markup for optimized discovery and user experience.
Writing Structured Content
While structured data formats allow labeling existing content as entities, properties etc, organizations may also directly author content in a structured format for optimal retrieval and display.
Some approaches to writing structured content include:
Database-driven content models: Content is entered into fields of a CMS, database or headless CMS like content types, and rendered programmatically on templates. Lends to consistent, structured outputs.
Template-based content: Authoring content as reusable blocks or components within layout templates designed for specific page sections ensures structured output.
Content modeling frameworks: Prominent ones being Topic Maps, DEF/DCTERMS, DocBook etc that provide document type definitions (DTDs) or schemas for semantic content markup across domains.
Natural language generation: Using AI to generate highly structured and consistent content from semantic topic models and databases without extensive human writing involved.
Content syndication standards: Formats like RSS, ATOM allow modeling feeds as structured content that is consistently consumed across platforms.
Authoring tools: Editors, styleguides and templates that guide writers towards semantic, structured composition helps standardize multi-authored content.
Structured content eases retrieval, display, analytics and future content reuse. It is crucial for technical documentation, databases, catalogs and other information meant for automated consumption. CMS UIs and style manuals help regular authors adopt structured writing processes.
Service Types and Modeling Services
As applications evolve towards microservices architectures, it becomes important to model services as first-class abstractions with well-defined interfaces, behaviors and contracts. This helps manage the growing complexity as well as enables portability, orchestration and governance of services.
Some common constructs for modeling services include:
API definition languages: OpenAPI/Swagger, API Blueprint etc define HTTP APIs in a structured format independant of implementation.
Interface definition languages: IDLs like Thrift, Protobuf, gRPC allow defining data types and RPC interfaces for language-neutral service contracts.
Service catalogs: Central registries to manage available enterprise services, their contracts, locations, documentation etc. May be integrated with API portals.
Service meshes: Tools like Istio provide service discovery, load balancing, authentication between microservices within a mesh.
Service graphs/maps: Visualizations of services as nodes and their dependencies as edges to understand architectures at aglance.
Behavioral models: Standards like BPMN, UML activity diagrams model service workflows, interactions and business flows beyond the technical interface.
Service level agreements: Non-functional contracts around guarantees for metrics like performance, uptime, response times etc.
Observability: Tools for monitoring service health, metrics and traces provide observability into a microservices environment.
Proper service modeling brings structure to otherwise complex, distributed systems. It forms the foundation for automation, governance and understanding as applications evolve over time with services.
Data and Service Model Orchestration
Once data models and services are defined, the next step is to integrate and orchestrate them as an overall solution. Integration platforms allow wiring together various structured components without hardcoded dependencies.
Some key pieces in orchestrating data and services include:
Application Program Interface (API) management: Central exposure of consistent, aggregated interfaces for internal and external consumers of services. API gateways and API management platforms allow this.
Service integration: Platforms like integration servers or iPaaS provide graphical workflows, connectors and tooling to wire services together without code. Handle data/payload transformation and routing too.
API-first development: Strict approach where APIs are designed and implemented first, followed by frontends/clients. Encourages loose coupling and contract-first development.
Enterprise Integration Patterns (EIPs): Standard integration architectures and best practices around common integration use cases relating to routing, transformation, workflows etc.
Data/Event streaming: orchestration engines like Kafka or Kinesis allow composing real-time event processing workflows from stream sources to consumption endpoints.
Backend for Frontend (BFF) pattern: Having dedicated backend services as front-facing APIs for each client/frontend microservice avoids overloading monolith backends.
Domain-Driven Design: Model domains as independent bounded contexts and utilize anti-corruption layers for inter-domain communication.
Proper orchestration middleware and patterns ensures coordinated evolution of structured services and applications over time with minimum tight coupling between independently deployable components. Integration-centric development is key to microservices architectures at scale.
Andrew Stroud