How the data is built, and what it can be relied on for.

The standing disclosures

From source record to screen

Nine steps sit between an agency's publication and what you search. Each one can introduce a difference between the record and the row, so each is described here rather than summarized as “processing.”

1. Collection

   We read 45 published sources on a fixed schedule — open-data APIs, agency portals, and public search systems. We read what the agency publishes; we do not scrape private systems, buy personal data, or obtain records through anything other than the public route. Each read is stamped, and the stamp is what the workspace shows you as freshness.

2. Normalization

   Source fields are mapped onto a consistent shape: dates parsed into real dates, boroughs and block-and-lot numbers into a single parcel identifier, statuses into the source's own vocabulary rather than a new one of ours. Company and person names are normalized for comparison — case, punctuation, and entity suffixes such as LLC or CORP — while the name as filed is kept and displayed. You always see what the agency wrote.

3. Deduplication

   Sources republish. Most agencies release a full dataset each cycle rather than a change feed, so the same record arrives many times. Records are keyed on the source's own identifier — a job number, a document id, a ticket number — and a repeat read updates that record instead of creating a second one. Where a source publishes no stable identifier, the key is built from the fields that identify the filing, and that is a design decision we will tell you about for the source in question.

4. Entity matching

   Two records naming the same company are grouped by comparing normalized names, and in some places by an identifier the sources share, such as a licence number or a PASSPort Supplier ID. An identifier match is reliable. A name match is not. Records are never merged into a single “resolved entity”: each keeps its own history and its own link back to the filing it came from.

5. Geocoding

   Locations come from the City's own address directory, which maps an address to a borough-block-lot and a building number. That directory is republished quarterly and carries no coordinates of its own, so a brand-new building can be permitted before it is listed. Records that cannot be placed are shown in lists and left off maps rather than being placed approximately. A map view therefore covers a subset of the records behind it, and says so.

6. Cross-source linking

   A parcel joins records across products reliably, because the parcel identifier is the same number in every city dataset. A company does not: no shared key exists between, say, a DOB filing and a procurement award, so those links are built on names. Trust the individual record over the page that groups it.

7. Refresh

   Every source has a published read cadence and a published expectation for how far behind the source's own data may run. Both are in the Data Coverage table. When a source falls outside its expectation, the workspace marks it delayed on the page rather than showing a stale figure as current.

8. Quality checks

   A read that returns implausibly few rows does not replace the data it would have overwritten. Row counts, the date a source runs through, and the time of the last successful read are recorded per source and surfaced in the product. Changes to existing records are found by comparing each read against the previous one, and corrections we make on our own side are excluded from those change feeds so that our fixes do not appear as agency activity.

9. Correction handling and retention

   Public records are themselves corrected and withdrawn. When a source revises a record we take the revision on the next read; when a source removes a record, we remove it too. We do not maintain a private archive of records an agency has retracted. Reports of a bad entity match are handled individually — see below.

Four kinds of field, and how to tell them apart

Not everything on a record page has the same standing. These four categories behave differently, and the difference matters most when you are about to rely on one.

How matching works, and its limits

Company, owner and party pages group records whose names match after normalization, and use a shared identifier where the sources carry one. A grouped page is a convenience, not a finding: open the individual records before you rely on a total. If a record is grouped with the wrong company, email hello@718systems.com with its job number, document id or ticket number and we will correct it.

What public records cannot tell you

Public records are incomplete, delayed, corrected after publication, and inconsistent between agencies that describe the same thing. This is a property of the records, not of the product, and no amount of processing removes it. Some consequences worth carrying into a decision:

• Absence is not evidence of absence. A search returning nothing tells you about the sources listed in Data Coverage. Work filed with a body we do not read, or in a legacy system an agency has retired, is simply not there.
• A status is as of the last read. Nothing on a page is a live query against the agency. A deadline, a licence, or an open balance should be confirmed at the source before you rely on it.
• Most sources record intent, not completion. A permit is authorization to begin. An abatement notification is filed before work. An accepted offering plan is not a building. Few feeds carry a verified completion date at all.
• A complaint is not a finding, and a violation is not a judgment about a current owner. Enforcement records attach to a building and a name, and can predate a sale.
• Amounts are as filed. Job values, quantities and estimated contract values are what the filer entered. No agency in these sources independently verifies them, and neither do we.

Questions about any of this

If a figure looks wrong, a match looks wrong, or you need to know whether a particular source is in scope before a demo, write to hello@718systems.com. Specific questions get specific answers — including “we do not cover that.”

See the source-by-source coverage tables →