March 2025: The end of paperwork

Sorry I couldn’t help myself

Regulated industries often run on “documentation”.[1] A simplified, non exhaustive way to look at this is that for each “transaction” (or customer interaction), the organization has to 

• Collect Information Step: collect certain information from counterparties (customers, vendors, partners etc)

• Rule Validation Step: validate that information against a set of rules, 

• Real World Reconciliation Step: Reconcile that information against something in the real world, 

• Signature Step: ensure the correct, authorized individuals sign certain documents (consents, terms and conditions etc)

  • The signers are typically taking on some kind of authority (clinical, statutory, fiduciary etc)

• Ongoing Monitoring Step: Ensure that on an ongoing basis, the behavior of each counterparty complies with 

  • What they agreed to at the start

  • Various statutes and regulations they’re subject to

• Exception Step: Take certain steps to fix exceptions 

  • And document that those steps were taken, and when, and any resolution

You can think of a “transaction” as the atomic unit of value a business provides. For a bank, a transaction might be opening a deposit account, opening a savings account, approving a loan, etc. For a healthcare company a transaction might be seeing a patient, submitting a claim, prescribing medication, etc. A customer relationship can span multiple transactions. In a regulated industry, for most mature transaction types, the required documentation is generally specified by law/regs, and relatively standardized (or otherwise within a reasonably narrow bound). 

Compliance is (colloquially) the function that ensures that the right information was collected, the right items were validated, the right monitoring is in place on an ongoing basis, and the exceptions were properly handled and documented. Audits are typically adjacent to compliance, and often involve sampling a number of transactions and reviewing the documentation collected.

Some examples:

I’m not as familiar with regulated industries other than healthcare and financial services, but my intuition is that the paperwork burden in energy, insurance, real estate, etc rhyme with the framework I’ve described above. One way to think about a lot of the paperwork/documentation is that the institution is converting information that is unstructured and/or received in a wide, broadly unpredictable variety of forms, into enough structure that it can be compared against a set of rules or policies. The institution then uses these rules to decide whether to enter into the relationship, how to price it, and so on.

As it turns out LLMs are good at turning multimodal unstructured data into a variety of structured forms (and vice versa). Some examples of this that I am extremely familiar with either because I directly helped build or have invested in these companies/products:

1. Carbon built tooling to extract visit intent from patient phone calls, report that to clinic managers, & drive visits

2. Abridge and the Carby AI scribes convert voice recordings of a patient visit into a chart and medical codes

3. Recon Health converts payer contracts into a) contracted rates so providers can know how much they will actually get paid and b) payor rules so providers know what to do to stay out of trouble

4. Inscope’s Report builder converts financial data into financial statements, footnotes and disclosures for private company filings.

5. Charta Health converts clinical documentation into auditable format

6. [I know this less well]. Abel converts police body camera footage into police reports

A general way to think of “documentation” in these industries is “proof of work” vs. “proof of thought”. 

Proof of work

Some types of documentation prove that you operated according to the rules. You can think of these as “proof of work”. Typically this documentation has a relatively standard format and is required. It’s audited on some cadence. The way to know that some kind of document is proof of work is whether it is default used as evidence when something goes wrong or when there’s a lawsuit. Medical records in healthcare and CIP/KYC data in banking are obvious examples. When something goes wrong with a patient, the medical record is the artifact that’s used by clinical leaders, auditors, lawyers etc (depending on the problem) to check if the patient received the standard of care and the clinicians did the “right” things. 

This type of documentation occurs much more frequently in regulated industries (or in heavily regulated parts of every industry, like HR or workplace safety). It is here to stay, will only really change when regulations change, and LLMs ultimately make these easier to generate, to audit, and make sure the documentation and the operational process they’re meant to document are actually occuring. The canonical example of this in healthcare is AI Scribes being used to generate chart notes/visit summaries. Visit notes are typically in a fixed format, and historically had to be generated by doctors (who are very expensive) by hand. AI scribes can generate a more complete version of a visit note, in the correct format, just from a transcript, with much less effort than a doctor could by hand. Over time, the burden of generating this type of paperwork is going to go away. 

Proof of thought

The other major type of documentation is more about enabling a discussion and  supporting a decision. These are meant to help the writer and the audience make high quality decisions, and prove that the writer was thoughtful about the different scenarios and edge cases the organization might face in executing on some decision . They’re not standardized, and they’re barely (if ever) audited, and once you arrive at the decision, you rarely return to these documents. These are things like product requirement documents (PRDs), RFCs & business memos. 

This type of documentation is really at risk, mostly because artifacts that used to mean that the author(s) had holistically thought through the problem, no longer dependably signal that the writer was thoughtful. A memo about a decision can now easily be generated by a consumer grade LLM, and in skimming it you would miss that the writer might not have thought through the decision as well as you (the audience, or the leader of the organization) would have expected them to. In proof-of-thought cases, downstream of an LLM generating text is a human who must synthesize meaning from it, critically think, and ensure it was prompted properly, and reflects the real risk that specific organization is taking in the real world as it goes about it’s business. As the cost of generating a lot of reasonable sounding text drops to zero, the downsides of a) the audience skimming and b) the organization relying on proof-of-thought artifacts rise exponentially. 

Almost perversely, the emergence and improvement of LLMs & multi modal models you can chain together will likely result in opposite effects for proof of work vs. proof of thought documentation. 

Proof of thought documentation will likely become less and less reliable over time, and organizations will have to develop antibodies to reduce the odds that large slugs of reasonable sounding text that are devoid of meaning infect their decision making processes. 

Proof of work documentation will likely get better and more complete, while the associated drudgery/tedium goes down. I think over time, the nature of white collar “work”, particularly in these regulated industries, can migrate from generating, managing and auditing paperwork, to ensuring that the real world outcomes the paperwork is supposed to produce, are actually produced, reduce the occurrences of exceptions (deviation from rules or policies) in real time, and ultimately match the institution/organization’s stated objectives. 

1. It’s likely lots more industries run on documentation, I’m just much less familiar with the purpose, why it exists, what its used for etc.