Migrating historical auction data faster with Basta

The goal was to move almost two decades of auction history onto the platform without losing the meaning behind the records, without exposing production data, and in far less time than the quarter it was estimated to take. This is where data engineering and agentic AI played a huge role in making that possible. Here is the short version of what came out of it:

• AI wrote the queries, people made the calls. The model read the schema and drafted the queries, our engineers ran them, and the team resolved the business context that was difficult to infer from the data alone and required auction expertise.

• A quarter of work landed in about a month. More than 1,100 auctions, 310,000 lots, almost one million bids and roughly $695 million in historical hammer value.

• Auction house data never left the house. The model worked against the schema and not the records, so no customer or production data was exposed to it at any point.

• The tooling outlived the migration. Moving an auction house onto Basta is faster today than it was before this project, and the same foundation allows us to explore setting clients up for self-service in the future.

Both the tooling and the experience from it have become part of how our subsidiary Basta works as more auction houses come onboard. If you are interested in auctions, data or agentic development, come along for the story of how it all came together.

The client had been in business for a long time and with that, their system accumulated a history of changes to both data and processes behind it. That is normal when different auction platforms have been used over time, each with its own way of structuring auctions and representing the domain.

So before starting the migration we had to work carefully with the client to decide what should be moved and how it should be represented. This is where the combined experience of Basta and Gangverk mattered as both teams have spent years building auction software. A deep understanding of auction workflows, lingo and the edge cases that come with them was already part of the team's expertise, making it much easier to understand how years of changes to the underlying systems were reflected in the data.

We gave Claude access to the application's code, database schema, migration history, existing integration code and our import API documentation. Using internal skills and some guidance from us on how Basta represents auction data, it was quick to trace relationships across the system and understand how fields had changed over time. This helped us build the transformation and checks needed to make sure vital information wasn't lost. However, the database contained a lot of information that we intentionally did not want to expose e.g. sensitive customer data and transactional history. Furthermore, the database was running live in production and we did not want an agent running freely against it, causing load spikes or possible downtime.

Thus we decided on the following approach: extract a backup of the database into a data lake, let Claude write out the queries based on the context which were then executed by our engineers. The results were then transformed by the tools that had been created earlier and the transformed data would be validated by stakeholders. That separation gave us the speed of AI-assisted development without giving the model access to personal auction data nor affecting live systems. All of this happened within a week.

The first results looked promising as most of the mapping to Basta was already in place. We verified the most recent auctions and most of it looked good. Until we went further back in time. That was where the final 20% began which ended up consuming most of the project time. The challenge was to understand what the data meant instead of simply mapping a field.

What is a winning bid? Is an absentee bid the same thing as a maximum bid? What does a cancelled bid mean when the old and new platforms handle cancellations differently? And if an older auction has a recorded hammer price but no individual bids, how should that sale be represented in the new system?

At a glance these can look like minor tweaks in data but on an auction platform they can change the history of who won, what a lot sold for, or whether it appears to have sold at all. We couldn't (nor wouldn't) risk importing this wrongly as that would skew historical analytics in Basta for our new client. Solving this required conversations with the people who understood the auctions.

That became an important part of the methodology. The agent explored the system and surfaced questions quickly, then we brought those questions to the people who can explain what the data actually represents. "Does this look right to you?" which translated into some adjustment which was fed back into the model to tweak the queries, the transformation logic and any cases of fallback methods needed to fit the Basta model.

We needed to be confident with our imports into Basta. So before moving data over we created a "dry-run" script for every auction to simulate how it would be moved over, which was then presented to a stakeholder. Once an auction had been reviewed and successfully moved over, we ran a reconciliation process that fetched the data back from Basta and compared it with what we expected to arrive. All this technical work would have taken weeks before we started working with agents, but since we had the methodology from earlier projects we could describe our ways of working to rapidly build the dry-runs, validations and reconciliation around it.

The result was a migration covering more than 1,100 auctions, 310,000 lots, almost one million bids and roughly $695 million in historical hammer value. Since the dry-run and reconciliation outputs were delivered with the migration, Basta now retains a history of how the record was transformed if anything ever needs to be traced back.

None of this would have been achievable without close collaboration with Basta throughout the migration. When we hit limitations in the import process or the client needed additional fields, the Basta team could often resolve it the same day. This allowed us to keep moving without losing momentum. Along the way we also surfaced a few issues in the import API, so those fixes made the import tooling better for the next migration onto the platform.

The research phase and getting into the mindset of the project happened in a fraction of the time it would normally take. That kind of a headstart allowed the Gangverk team to spend more time on the parts that required judgement such as understanding the business or validating assumptions.

Agentic development made it cheap to build tools and documentation that would normally have been hard to justify for a single migration. The same foundation can now be applied for future migrations instead of starting from scratch. Some might think that an agent in a /loop would solve this automatically but our experience was the opposite. The mechanical parts were faster only when the team kept resolving business context and decisions it couldn't infer from the data alone.

Moving the records was the easy part. The real work was understanding historical auction representation well enough to know how it should arrive on the other side. AI gave us considerably more time to focus on exactly that.