A financial statement describes a period in a company's history, but that does not mean investors could read it at the end of that period. For a research pipeline, confusing those two dates can quietly put later information into an earlier decision.

Misha Dvorkin, an analyst on our Security-Specific team, built a dataset that makes the distinction explicit. I asked Misha to focus on a practical question: when we inspect a historical signal, can we explain which disclosures were available at the time? He owned the data model, selection rules, and checks that turn that question into something the pipeline can answer.

Give each observation more than one date

The first change was to stop treating the reporting-period end as the only date attached to a value. Misha's records keep the period being described, the filing reference, and the availability information needed by the research workflow.

The SEC's EDGAR API documentation describes both filing-history data and financial-statement facts. Misha used that distinction to organize the pipeline: a reported value and its disclosure context belong together, even when they arrive through different parts of the ingestion process.

Each research observation also retains its source identifier. A reviewer should be able to move from a factor input back to the record that supplied it, without reconstructing the join from memory.

Make the selection rule explicit

For a given research date, the pipeline selects an eligible observation rather than simply taking the newest value in the database. The rule uses the configured information cutoff and records why a particular disclosure qualified.

We kept the cutoff convention separate from the factor calculation. That lets the team review how publication timing is handled without also changing the financial definition of a signal. It also makes comparisons between two data policies easier to understand.

Misha kept later amendments as separate records instead of overwriting the earlier observation. A correction can be useful information while still being information that was unavailable to an earlier experiment.

Resist convenient substitutions

The difficult cases were rarely clean, complete filings. Some companies had missing fields, different reporting calendars, or multiple records that looked interchangeable until we inspected their context.

Misha chose to flag ambiguity rather than silently select whichever record happened to appear first. The output includes both the selected observation and a reason code for observations excluded from a particular research date.

That creates a tradeoff: fewer rows may qualify for a given experiment. We accepted that cost in exchange for a dataset whose selection rules can be inspected. A larger dataset is not automatically a better one if its extra rows change what the experiment was allowed to know.

Test the boundary of availability

Misha built fixtures around the moments where a disclosure changes eligibility. A record must remain unavailable before its cutoff and become eligible only when the policy permits it. Adding a later amendment must not change an earlier research snapshot.

He also checked that reordering source records leaves the selected result unchanged. The important behavior belongs in the selection policy, not in the accidental order of a downloaded file.

What Misha owned

Misha owned the observation schema, filing joins, eligibility rules, and validation fixtures. He also wrote the inspection table that explains which records contributed to each signal input.

The result is a clearer foundation for the Security-Specific team's research. Before arguing about a factor's economic interpretation, we can first establish what information the experiment actually used. That is a small question to ask, but an essential one to answer well.

More from Security-Specific