Nobody Cross-Examines the Tool.

Between 2000 and 2015, Post Office records identified hundreds of private prosecutions arising from Horizon-era cases. Evidence to the statutory inquiry at one stage recorded 844 prosecutions resulting in 705 convictions, although later reconciliation changed some of those figures, and across the UK more than 900 convictions have been associated with Horizon evidence once other prosecuting authorities are counted. Horizon accounting data was central or essential in many of those prosecutions. It showed a shortfall, the shortfall was treated as a fact about the world, and people lost their businesses, their savings, their marriages, and in some cases their liberty. Several died before it was sorted out.

In 2019 a High Court judge finally examined the system itself and found it had "bugs, errors and defects," that the early version of it was "not remotely robust," and that there was a "material risk" the disputed shortfalls had been caused by the system rather than by the people accused. In his words, Horizon was "therefore not 'extremely unlikely' to be the cause of shortfalls in branches, as contended for by the Post Office." Those findings attach to the iterations in use from 2000 to 2017. He treated the version that followed as substantially sounder, though the statutory inquiry has since cautioned that this assumption may no longer be wholly justified in light of later evidence. The Court of Appeal began quashing convictions in 2021. Parliament eventually had to legislate to quash convictions in bulk, one act for England, Wales and Northern Ireland and a separate act of the Scottish Parliament for Scotland, because the ordinary appellate machinery could not move fast enough to undo what had been done on the strength of computer output nobody had validated.

I bring up a British accounting system in a newsletter for investigators because the failure was not technical. Plenty of people inside that organization knew Horizon had problems. The failure was that a machine's output was allowed into a courtroom carrying more authority than any witness had earned for it, and nobody in the room was equipped to ask the only question that mattered. Not "what does the system say." How do you know the system is right?

Our field has the opposite starting position and the same exposure. The commercial forensic tools are genuinely good. They are tested, documented, and improved constantly by people who are better engineers than I will ever be. That quality is what makes the exposure so quiet. A tool that is right almost every time trains you to stop checking, and you will not notice the habit forming.

What a vendor certification actually certifies, and why you still need one

I want to be careful here, because this is the part that gets misread.

Vendor certifications are worth the money and the week. They are the fastest path to competence on a specific piece of software, they are taught by people who know the product's internals, and they end in an examination with a defined pass condition rather than a certificate of attendance. If you work mobile, computer, or cloud extractions and you have no vendor training, get some. Nothing below is an argument against that.

I hold one. I am a Magnet Certified Forensics Examiner, and the exam earned my respect. I also run Cellebrite extractions and examinations regularly and hold no Cellebrite certification of any kind, which I say out loud because it makes the point better than an argument does. Neither of those facts is what opposing counsel asks me about. Across twenty-seven years in law enforcement, nineteen of them in digital forensics, and more than two hundred cases, I have never once been asked which vendor course I completed. I have been asked, many times and in several different accents of hostility, how I know what I claim to know.

What a vendor certification establishes is that you demonstrated competence against that vendor's defined training and examination requirements. That is real, and it is necessary, and it is roughly a third of the job. It does not by itself establish what the artifact you just recovered actually means, whether the tool interpreted it correctly in this case, what it cannot see, or whether your case-specific conclusion is a finding or a guess wearing a finding's clothes. Those are not gaps in the vendor's curriculum. They are outside its scope. The vendor teaches you their product. Nobody in that classroom is responsible for teaching you to survive a Daubert challenge, and it is unfair to expect it of them.

What the courtroom actually asks

Federal Rule of Evidence 702 was amended effective December 1, 2023, and the amendment matters more to working examiners than the coverage suggested at the time.

Two changes. First, the rule now makes explicit that the party offering the expert has to show the admissibility requirements are more likely than not satisfied. The Advisory Committee said plainly why: many courts had been treating reliability as a question of weight for the jury rather than a gate for the judge. Second, and this is the one to write on your monitor, subsection (d) now reads that "the expert's opinion reflects a reliable application of the principles and methods to the facts of the case."

Reliable application. To these facts. By you.

Read that against Kumho Tire, where the Supreme Court held that the gatekeeping obligation covers not just scientific testimony but technical and other specialized knowledge, including experience-based expertise, and described the objective of that obligation as making certain an expert "employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field." 526 U.S. 137, 141, 152 (1999). The tool's rigor is not admissible by proxy. Yours is the thing being examined.

There is a separate and often-missed piece under Rule 901(b)(9), one of the rule's illustrations rather than an exhaustive test, which allows authentication by "evidence describing a process or system and showing that it produces an accurate result." That obligation belongs to the proponent, and in practice the foundation comes through the examiner, another qualified witness, testing, documentation, or some combination. Authentication is also not the same question as Rule 702 reliability. But notice who is not carrying any of it: the vendor, who is not in the room and has no duty to your case. The software's marketing material is not a foundation, and a certificate with your name on it does not describe a process or show that it produced an accurate result here.

This is the whole asymmetry. The tool has no obligations. It will not be sworn, will not be impeached, and will not be asked to explain itself. Every duty the law imposes lands on the human who signed the report.

Worth knowing that the rulemakers have noticed. Out of its work on artificial intelligence and machine learning, the federal Advisory Committee on Evidence Rules published a proposed new Rule 707, titled Machine-Generated Evidence, for public comment between August 2025 and February 2026. As issued, it read: "When machine-generated evidence is offered without an expert witness and would be subject to Rule 702 if testified to by a witness, the court may admit the evidence only if it satisfies the requirements of Rule 702(a)-(d)." It carved out the output of simple scientific instruments.

Two things about that are worth your attention. The committee's own note explains that where a testifying expert relies on a machine method, the method and the reliance on it get scrutinized under Rule 702 already, and the proposal exists to stop a proponent evading those requirements by putting the machine output in directly, or through a witness "who applied the program but knows little or nothing about its reliability." The committee also concluded that in the cases Rule 707 would reach, it renders Rule 901(b)(9) irrelevant, which tells you how the people drafting these rules rank authentication against reliability.

Do not cite it as authority. It remains under consideration and has not been adopted: the committee declined to advance it, revised it, and held it for further study alongside the separate problem of deepfakes. But the direction of travel is not ambiguous, and the gap this issue is about is the one they are trying to close.

The five questions a vendor course was never built to answer

These are the questions I would put to my own work before anyone else got the chance.

1. What does this artifact actually mean? Your tool parses a database and prints a label. "Deleted" in a results pane is usually a flag in a table, not a statement about what a person did. A timestamp is written by some process, at some moment, in some timezone, and may be settable by the user or resettable by a sync. Knowing what writes an artifact, what else could write the same trace, and what an innocent explanation looks like is the actual skill. The tool's label is the beginning of that analysis and is routinely mistaken for the end of it.

2. How do you know the tool got it right in this case? Programs like NIST's Computer Forensics Tool Testing effort test tools against published specifications, which is valuable and is not the same as your validation, on your build, against your data types, documented in your own file. When a finding carries real weight, verify it by a genuinely independent method: the underlying data by hand, or a tool that does not share the first one's parsers. Two commercial products can inherit the same library and the same assumption, which makes agreement between them weaker evidence than it looks. If you never checked at all, you have a screenshot.

3. What are the limits and the failure modes? Parsers have versions. Application formats change without notice and the parser catches up later, which means the same extraction run on two builds can produce two answers. Read the release notes, know which build produced your output, and be able to say what your process cannot see. Daubert's list of reliability considerations includes the known or potential rate of error for a reason, 509 U.S. at 594. "I do not know what it misses" is an answer that ends badly.

4. Where did your authority end? Scope is not a technical question and it will sink technically perfect work. What did the warrant, the consent, or the engagement letter actually permit, and does your extraction stay inside it? A full physical extraction on an authority that covered a narrow category is an argument you handed over for free, though acquisition scope and review scope are not the same thing and what follows from exceeding either is jurisdiction and fact specific.

5. Can you explain all of it out loud, without the report in front of you? If your understanding of a conclusion lives inside a tool's user interface, you do not own that conclusion yet, and the witness box is where you find that out.

What this looks like when it goes wrong

It is never dramatic, which is most of the problem. Consider a hypothetical, because I do not want anyone hunting for a citation that is not there.

An examiner runs a clean extraction with a current build. The tool surfaces a location record and labels it clearly. The report says the device was at a location at a time, because that is what the pane said. Under cross, someone who did their homework establishes that the record in question is written by an application cache rather than the device's own positioning, that it can populate from a network lookup rather than a GPS fix, and that the examiner never distinguished between the two.

No fabrication. No misconduct. A competent operator produced a conclusion the underlying data did not support, and the entire chain from device to testimony was technically correct except for the interpretive step that no vendor course covered. Once that lands, the rest of the report is fair game, and so is everything else that examiner has ever filed in that jurisdiction. Opposing counsel will go back through the earlier work looking for the same interpretive step, and they will usually find it, because it was a habit rather than a one-off mistake.

The protocol, in one box

Before you touch the device:

• Read the authority document and write down, in your notes, the boundary of what you may collect.
• Record tool names and exact build numbers. Hash static images and files, and where a live, cloud, API or logical acquisition gives you no stable source-side hash, write down that it did not and why.

While you examine:

• For every artifact you intend to rely on, write down what process creates it and what else could create the same trace.
• Verify anything load-bearing a second, independent way.
• Note what your method cannot see, while you still remember.

Before you sign:

• Separate observation from inference, in the language of the report itself. "The database contains" is not "the user did."
• Attach a confidence level to every inference and be prepared to justify it. That is my own practice rather than a requirement of any standard, and I would avoid numeric confidence values unless you can support the number.
• Reread your conclusions as an opponent. Where is the interpretive leap?
• Ask yourself the question from the top of this issue about your own work. How do you know?

Bottom line

Get the vendor certification. It is the fastest route to operating competence and there is no substitute for it. Just be clear about what you bought. You bought the ability to run the instrument. What gets tested in a deposition is the layer above the instrument: what the artifact means, how you verified it, what you cannot see, whether you stayed inside your authority, and whether you can defend the reasoning without the screen. That layer is not in the product manual, because it is not about the product.

The tool will never be sworn in. You will.