Semi-structured data

Semi-structured data is data that carries its own labels, such as field names or tags, without following a fixed table design. JSON, XML, CSV exports, and most log files are semi-structured: a program can read their fields, but each record can differ, and the data usually lives in files rather than in a database.

Key takeaways

Semi-structured data labels itself, but does not follow a fixed schema.
It usually lives in files, so it is stored and managed like unstructured data.
Logs, sensor output, and data exports make it one of the fastest-growing kinds of data.

Why does semi-structured data matter?

Because applications and machines produce more of it every day: logs, sensor readings, exports, and the records that systems exchange with each other. It holds useful detail for analytics and AI, yet it usually sits in files spread across storage systems, so it needs the same care as other unstructured data: knowing where it is, who owns it, and how long to keep it.

What are examples of semi-structured data?

Common examples include:

JSON and XML files, used by applications to store and exchange records.
CSV files and spreadsheet exports from business systems.
Log files from servers, applications, and devices.
Sensor and device output, such as readings from vehicles, instruments, and equipment.
Email, where the sender, date, and subject are labeled but the message body is free text.

Is JSON structured or semi-structured data?

JSON is semi-structured. Each value has a label, so a program knows what it is reading, but records in the same file can have different fields, nested levels, or missing values, and nothing forces them to follow one table design. Databases can store JSON, but the format itself does not require a schema.

Is a log file structured data?

Most log files are semi-structured. Each line usually follows a pattern, such as a date and time, a level such as error or warning, and a message, so tools can read the parts. The message itself is often free text, and formats change between applications and versions, which is why logs are harder to analyze than database records.

How is semi-structured data stored?

Mostly as files, on file systems and object storage, often in very large numbers. Some is loaded into data lakes and lakehouses for analysis, or into databases that support JSON and documents. The original files usually stay where they were written, so the same data often exists in several places.

How do you manage semi-structured data?

The same way as other files: index the metadata of every file, such as path, size, owner, dates, and type, so you know where it all sits and how fast it grows. Then apply rules, such as retention periods for logs, which pile up quickly, and select the right files for analytics or AI. Format-aware tools can read the fields inside when a project needs that detail.

How does Diskover help?

Diskover indexes semi-structured files, such as JSON, XML, CSV, and logs, alongside every other file, so teams can find, size, and manage them by owner, age, and project.

Explore the Diskover platform

Book a demo