Dataset architecture / evidence layers

Datasets

Why the research uses several data layers instead of one giant table.

The research program contains almost 1.8 million discussion URLs. That scale is useful. But it is not enough for deep analysis. Each layer answers a different question.

Abstract concrete geometry
DATASETS
UNITS
CONTEXT
TRACEABLE EVIDENCEInventory
validation
gold dataset
derived layers
1,779,805inventory URLs
5,000validation threads
1,000Gold Dataset threads
536semantic relationships
1,493,085macro subset threads

The research program contains almost 1.8 million discussion URLs.

That scale is useful.

But it is not enough for deep analysis.

A large inventory can tell us how much material exists.

It cannot, by itself, tell us whether a visible link is an explicit Brand Mention, whether wording repeats or how many semantic relationships exist inside one passage.

That is why the research is divided into separate data layers.

Each layer answers a different question.

The research works as a chain of evidence layers

Inventory → Validation → Gold Dataset → Derived datasets → Graphs and findings. Each step adds depth and narrows the analytical population.

01
Full Research Inventory

1,779,805 unique discussion URLs

The Full Research Inventory is the scale layer of the project.

It contains 1,779,805 unique discussion URLs from approximately ten years of FunkyMEDIA-owned publishing infrastructure.

Its purpose is to define the source pool.

The Full Research Inventory supports sampling, large-scale structural analysis, and the macro view of the publishing environment.

It is not a count of Brand Mentions.

A discussion URL is the inventory unit.

A Brand Mention has to be identified inside the text.

Open Corpus v1 →

02
Representative Validation

5,000 threads / 8,708 posts

The Representative Validation layer was created to inspect the shape of the wider inventory without attempting deep semantic classification of every record.

It contains 5,000 randomly sampled threads and 8,708 posts.

This layer is used to check chronology, structure and the general form of the source material.

It is separate from the Gold Dataset.

Validation tells us whether the sampled material reflects the wider inventory structure.

It does not replace deep analysis.

03
Gold Dataset v1

1,000 threads / 4,447 posts / 794 outbound links

Gold Dataset v1 is the deep-analysis layer.

It contains 1,000 threads, 4,447 posts and 794 outbound links.

This is the dataset used for the public Brand Mention taxonomy, repetition analysis, intent calibration and semantic relationship extraction.

The Gold Dataset is smaller than the Full Research Inventory because its purpose is different.

The inventory provides scale.

Gold Dataset provides depth.

Open Gold Dataset v1 →

Three public layers derived from Gold Dataset v1

These layers are not new populations. They are structured classification layers built from the deep-analysis dataset.

Brand Mention Taxonomy v1

Linked, unlinked and entity-reference classes

This derived dataset stores the structural classification behind one of the main findings.

It separates Explicit Brand Mention — Linked, Unlinked Brand Mention and Linked Entity Reference.

The public Gold Dataset calibration contains 416 linked explicit Brand Mentions, 328 unlinked Brand Mentions and 378 Linked Entity References.

The taxonomy is useful because it keeps visible naming and hyperlink presence separate.

It is not a new population.

It is a classification layer derived from Gold Dataset v1.

Open Brand Mention Taxonomy v1 →

Formulaic Language v1

Repetition inside explicit Brand Mentions

This derived dataset contains the repetition classification for the 744 explicit Brand Mentions in Gold Dataset v1.

The current results include 63 exact formulaic mentions, 21 exact formulaic clusters and 681 unique-like mentions.

This dataset is used to separate frequency from language diversity.

The term unique-like does not mean organic.

It means only that the current method did not classify the wording as an exact or near-text duplicate.

Open Formulaic Language v1 →

Semantic Relationships v1

Brand → Relation → Entity records

This derived dataset contains the semantic relationship layer extracted from Gold Dataset v1.

The current calibration contains 536 relationship records, 276 unique Brand → Relation → Entity pairs, 139 brands with at least one detected semantic relationship and 98 entities or relationship terms.

The relationship families include Use Case, Product, Service, Feature, Problem, Result, Location and Co-Mention.

This dataset is the foundation of Brand Context Graph and the related graph views.

Open Semantic Relationships v1 →

04
Macro Subset

1,493,085 offline threads

The Macro Subset is used for large-scale title, category and repetition analysis.

It contains 1,493,085 offline threads.

This layer produced structural findings such as 1,036,637 threads — 69.43% — in review/recommendation-oriented paths or categories, 307,749 opinion-oriented titles — 20.61%, 308,715 recommendation-oriented titles — 20.68%, 57,609 question-mark titles — 3.86% and 127,123 titles containing company/business wording — 8.51%.

It also produced title-level repetition findings.

These are structural title and path observations.

They are not explicit Brand Mention counts.

A number should remain attached to the population that produced it.

1,779,805 is an inventory count.

744 is an explicit Brand Mention count inside Gold Dataset v1.

536 is a semantic relationship count.

1,493,085 is a macro subset thread count.

Those numbers describe different units.

Combining them into one headline would make the research look larger while making the interpretation worse.

The purpose of the dataset structure is to prevent that.

Match the dataset to the question

Use the Full Research Inventory when the question is scale.

Use Representative Validation when the question is structure and chronology.

Use Gold Dataset v1 when the question requires deep classification.

Use Brand Mention Taxonomy v1 when the question is naming versus linking.

Use Formulaic Language v1 when the question is repetition.

Use Semantic Relationships v1 when the question is contextual relationships.

Use the Macro Subset when the question concerns large-scale title, category or repetition structure.

The dataset should follow the research question. Not the other way around.

What the datasets do not claim

The Full Research Inventory does not claim to represent the entire internet.

Gold Dataset v1 percentages are not presented as full-corpus prevalence.

The Macro Subset does not provide explicit Brand Mention counts.

Unique-like wording does not prove organic origin.

Semantic relationships do not automatically represent verified real-world facts.

And dataset size does not, by itself, make a conclusion stronger.

The value of the data comes from matching the correct unit to the correct question.

The goal is not to make every number as large as possible.

The research program is easier to understand if the datasets are viewed as a chain.

Inventory → Validation → Gold Dataset → Derived datasets → Graphs and findings

Each step adds depth.

Each step also narrows the population.

That is intentional.

The goal is to keep every finding traceable to the dataset that actually supports it.