Before course-based study existed, we had a recommendation system that I can only describe, looking back, as guessing with extra steps. A student opened the app, and somewhere in the backend a query ran that matched their department to a book's department, their school to a book's school, their level to a book's level. If that turned up nothing, it dropped the level and tried again. If that still turned up nothing, it dropped the school too and just matched on department. It always returned something. That was the whole design goal, honestly: never show an empty page. Whether what it returned had anything to do with what the student was actually studying that week was a separate question, one the system had no way of answering, because it had never once looked at what any of the books or courses actually contained.
I knew this couldn't be the foundation of anything real. So in the middle of March, we started over, and this is the story of what that turned into: not a clean rebuild that went smoothly, but a year-long argument with our own data, one we're still not finished having.
Here's the shape of it, before I walk through it properly.
Seventeen PDFs and half a million lines
The very first commit to what became the course-based-study repository isn't technically mine. Forty minutes before it, a Copilot coding agent committed something called "Initial plan," and I've kept that in the history on purpose, because it's true to how this actually started: I didn't sit down and architect a data pipeline from scratch. I handed the problem to an agent and asked it to figure out where to begin, then took it from there myself once there was something to react to.
That first real commit, on March 16, already carried thirteen university course catalogs pulled straight from Nigeria's Core Curriculum and Minimum Academic Standards documents. Administration and Management. Agriculture. Allied Health Sciences. Architecture. Arts. Computing. Education. Environmental Sciences. Law. Medicine and Dentistry. Sciences. Social Sciences. Veterinary Medicine. Four more joined over the following weeks, Basic Medical Sciences, Communication and Media Studies, Engineering Technology, and Pharmacy, until we had all seventeen. Sixty-three megabytes of scanned academic bureaucracy, each catalog its own PDF, each one apparently designed by a different committee with a different idea of how a course listing should look.
Running text extraction across all seventeen produced a single file, 549,734 lines long. I remember opening it for the first time and just scrolling, because there was no other way to get a feel for what we were dealing with. Somewhere in that half a million lines was the entire structure of a Nigerian university education: every programme, every course code, every learning outcome anyone had bothered to write down. If we could get a machine to actually read it the way a person would, we'd have something nobody else had built. Getting there took days, and not because pulling text out of a PDF is technically difficult. It's because every catalog used its own conventions. One department wrote course codes with a colon after the number. Another left it out entirely. A third split the title across two lines whenever it ran long, so a naive parser would read half a course name and quietly stop. Multiply that by seventeen documents and you get a problem that looks like formatting and is actually closer to archaeology.
Four books, one at a time
Before we built anything resembling a pipeline, we needed to answer a much smaller question first: does matching a book to a course by what it's actually about, instead of by a label on someone's profile, even work? We didn't build infrastructure to find out. We picked four books we happened to have lying around, a data communications textbook, one on Prolog, a chemistry book, and something on ontology, and ran each one by hand through a script that embedded its content and searched for the nearest course descriptions. No batching. No pipeline. Just four attempts, watched closely, to see if the core idea held up before we spent another month building around it.
It held up well enough. Not perfectly. Well enough to be worth continuing, which in a project like this is most of what you can ask for at that stage.
The map, and the cracks we walked past
By early April we had something real: a programme_courses.json file mapping every course code to the degree programme, level, and semester it belonged to, generated straight from the seventeen catalogs. It covered 202 of 224 programmes on the first pass, which meant the next two weeks were spent chasing the other 22 and fixing what the first attempt had gotten wrong along the way. We corrected how a course's level got inferred from its own code. We isolated a set of education courses that had drifted into the wrong programme's list. And on April 15, two commits landed with names that, months later, I'd recognize as the first real warning we ignored: one titled simply "isolate education catalogue courses," the other "add spillover warnings."
At the time these felt like ordinary bug fixes, the kind you write, merge, and forget by lunch. What we were actually looking at, without realizing it, was the edge of a much bigger pattern: two unrelated departments had independently chosen the same short course-code prefix for entirely different subjects, and our extraction logic had no way to know that was even possible, let alone handle it correctly. We fixed the specific case sitting in front of us. The pattern underneath stayed exactly where it was, untouched, waiting.
Shipping on faith
On May 18, the production matching engine went live in a single commit, 1,406 lines, fully built, running against the same catalog we'd mapped in April. Give it a book, and it would embed the content, search across every course we knew about, and ask a language model to confirm which matches were actually strong enough to keep. It worked immediately, in the sense that books started attaching themselves to courses with no human tagging anything by hand, which felt, at the time, like the whole point had been achieved.
A few weeks later we tightened the eligibility rules so a handwritten scan or an old exam paper wouldn't get treated the same as a proper textbook. Then the project went quiet for two months. Not because we stopped caring, but because it was working, and working systems have a way of not asking for your attention.
The catalog underneath it had never once been audited for the exact failure mode those two April commits had brushed past. It had been running in production the entire time, quietly wrong in ways nobody had gone looking for.
The afternoon I found my own mistake
In August I was in the app, looking at my own course list under one of the computing programmes, and there it was. Aviation and Space Insurance, sitting where a project management course was supposed to be. Not a display bug. Not something stale in a cache somewhere. The actual data, for six different computing-adjacent programmes at once, all quietly pointing at an insurance elective none of us had any business seeing.
I want to be honest about what that moment actually was, because it would be easy to write it as an outside report, a user flagging something, a support ticket that got escalated. It wasn't. I built the system that did this. I'd been looking at my own course list, in an app I'd shipped months earlier, and only noticed because I happened to know what should have been there instead. That's the part that stayed with me: I hadn't broken something loudly enough to notice. I'd built something that had been wrong from close to the beginning and had simply never said so.
What a silent collision actually is
The mechanism, once I understood it, was almost embarrassingly simple. Two departments, entirely unrelated to each other, had each picked the same short prefix for their own course codes. It's not really a coincidence once you think about how many departments there are and how few good three-letter combinations exist. ARC meant Architecture to one department and Archaeology to another. INS meant Insurance to one and Information Systems to five more programmes that cross-listed it. Every time two departments' numbering happened to land on the same code, our pipeline had to choose a winner, and it chose one by accident: whichever course's extracted description happened to be longer won, and the other simply disappeared from everywhere downstream that mattered.
Here's what that actually looked like for one code, ARC101, the one that happens to make the clearest picture:
Nothing crashed. Nothing logged an error. Every page still loaded, every course still had a title, and that's exactly what made this the worst kind of bug I've dealt with. A crash tells you, immediately and unambiguously, that something is broken. A silent collision tells you the opposite. It tells you everything is fine, in a system that answers every request correctly by every automated test we had, while quietly lying about a slice of its answers that no test had ever been written to catch, because nobody had known to write one.
Going through 210 of them, by hand
We built a scanner that compared, for every course code, which department structurally claimed it against every distinct title that code had ever been given anywhere across the seventeen catalogs, grouping near-identical rewordings of the same course together so that re-typed variants didn't get counted as false alarms. What came back was 210 codes carrying this exact pattern.
Architecture and Archaeology alone accounted for thirteen of them. Line 120,429 of the merged catalog text reads ARC 101: Introduction to Architecture. Line 135,288, fifteen thousand lines later, reads ARC 101: Introduction to Archaeology. Both are real courses, taught to real students, in departments that had no reason to know about each other's numbering. Only one of them could keep the code, and for months, the choice had been made by accident.
Fixing all 210 properly meant adopting one rule and holding to it without exception: never trust the clustering script's confidence on its own. Every single fix got traced back to an actual line number in the source text, and where the extracted text itself turned out unreliable, which happened more than once, we went back to the original PDF and read the exact page its own table of contents pointed us to. One department's catalog interleaved two adjacent programmes across a stretch of twenty-five thousand lines with no clean boundary between them anywhere, and that one only got resolved by opening the PDF directly and reading it like a person would. It was slow, deliberately so, because the whole reason we were doing this by hand instead of trusting the algorithm was that the algorithm was exactly what had gotten us here in the first place.
Two mistakes we caught ourselves
Partway through the investigation, a manual correction to one course title, patched by hand instead of through a proper file writer, left an unescaped comma sitting in a place it shouldn't have been. It corrupted one of our data files mid-write, truncating what should have been six thousand rows down to just over three hundred before the script writing it crashed outright. We caught it within minutes, because by that point we'd made a habit of cross-checking row counts across every output file after every batch, and we rebuilt the missing rows from an untouched copy of the data that a later step in the same failed run hadn't gotten around to overwriting yet. Nothing was actually lost. But it was a sharp, immediate reminder that a fix applied carelessly can do the exact kind of silent damage we were there trying to undo, and that the discipline we'd adopted for finding other people's mistakes needed to apply just as strictly to our own.
Months later, while carrying all 210 fixes into the live system, we made a second mistake that mattered more. Renaming a course code in our own records was only half the job. Real books, already attached to the old code because of a real earlier run of the matching engine, needed to move to the new code too, but only in the cases where the old code had genuinely been showing the wrong content the whole time. The first version of the script that did this didn't check for that condition carefully enough. It ran against every renamed code indiscriminately, not just the ones that had actually been wrong, and started copying books that had never been misattributed at all onto courses they had nothing to do with. We caught it after roughly two hundred and sixty books had already moved, not zero, but small enough, and because the underlying database lets you trace exactly where each book came from, every single one of them got moved back to where it belonged before it ever reached a real student's screen.
Both times, the lesson landed the same way: verify after every batch you run, not only before you start.
The day it fell over
The clean version of this story ends there, with two caught mistakes and a tidy sense of having learned something. The honest version doesn't. When we started rolling the fixed catalog into the live system, something we hadn't fully anticipated happened: one correction touched data that another part of the system was relying on in a shape it no longer expected, and the errors that produced didn't stay contained. They cascaded. A fix in one place surfaced a fault somewhere else, which surfaced another, and within a short window we had enough real, user-facing errors piling up on top of each other that trying to patch it live, one symptom at a time, stopped being a responsible option. We had to shut the server down and work through the actual chain of failures properly, offline, rather than keep chasing individual errors while the app stayed up. That cost us roughly a day. Not a metaphorical one. The app was genuinely unreachable, we knew exactly why, and getting it back up correctly mattered more to us than getting it back up quickly. It's the part of this story that's easiest to leave out of a tidy writeup, and the part I think matters most to include honestly.
Rebuilding the engine on ground we could finally trust
With the catalog actually verified, we went back to the matching engine itself and found a second, quieter problem waiting in how it searched. It had only ever worked in one direction: take a book, search every course, keep whatever matched well enough. That's the right instinct for the common case, a new book arriving and needing somewhere to go. But it has a blind spot, and Nigerian university catalogs are full of exactly the kind of course that falls into it: narrow, small-enrollment, easy to overlook. A niche course can lose the competition inside any one book's individual ranking even when that book would have been one of the best matches available for it specifically, simply because more generically popular subjects keep crowding it out of that particular book's shortlist. The course never gets its own turn to ask the question.
So we built the other direction too.
Now, for each course, we can search across every eligible book on its own terms, without competing against every other course's claim on the same shelf. Both directions run against the same confidence thresholds and write results the same additive, idempotent way, so a book earning a 0.87 confidence score means the same thing regardless of which direction found it. We tested it against five real courses before trusting it with anything live, and one of them came back with nothing at all: a ceramics workshop course in Industrial Design, searched properly and given every chance, genuinely had no confident match anywhere in the current book library, because nothing in that library is actually about ceramics. That's not the search failing. That's the search telling us the truth about a gap that's real, which is precisely what we needed it to be capable of doing, even when the honest answer is one we'd rather not hear.
Where this stands
Right now, when a new course gets added to the catalog, it gets its own dedicated search for the books that actually belong to it. When a new book comes in, it gets sorted into the courses it genuinely fits. Both directions run in production, and both check their own work against a catalog that's finally been read closely enough, all seventeen documents of it, to be worth trusting.
It isn't finished. There are courses we still don't have good books for, and corners of the catalog we haven't stress-tested the way the loudest failures forced us to stress-test the rest. This is ongoing work, and I'd rather say that plainly than let a data pipeline sound more finished than any data pipeline actually is.
But I believe the direction is right. Read the source material properly instead of trusting a shortcut. Catch your own mistakes before someone else has to. Verify instead of assume, every single time, even when it's slow, even when it means taking the server down for a day to do it properly. I think that's genuinely what it takes to turn the scattered, department-by-department, differently-formatted chaos of the Nigerian university book system into something a student can open and simply believe.