Can software be replaced without changing what it does?

The problem

Why this matters outside computer science

Public institutions run software that outlived its builders: benefits calculations in COBOL, hydrological and traffic models in Fortran, case management in VB6 or Delphi. These systems encode decades of policy, exceptions and corrections that exist nowhere else — not in the documentation, and not in anyone's memory.

The barrier to replacement is therefore not engineering capacity but epistemic: nobody can state with confidence what the current system does, so nobody can prove a replacement does the same. Organisations stay with a vendor or a language because leaving cannot be shown to be safe. Software independence fails on a verification problem.

Research question & method

Verified legacy migration

How reliably can agentic AI migrate a legacy codebase to a modern stack while preserving its observable behaviour, and where exactly does it fail?

1. Characterize Derive the behaviour of the running legacy system from real inputs, outputs and state transitions — not from documentation — and express it as an executable characterization-test suite [1].
2. Migrate Run multi-agent workflows that translate the codebase to a modern, open target stack, with the test suite withheld from the migrating agents.
3. Verify Run both systems against identical inputs and compare results case by case — differential testing [2] — including edge cases, error paths and numeric tolerance.
4. Measure Record equivalence rate, failure taxonomy, human interventions required, and cost per thousand lines — per language, per system size, per model.

Scope

Only publicly available legacy codebases are used, so results are reproducible and nothing proprietary or confidential enters the work. Target languages are chosen for public-sector relevance: COBOL, Fortran, VB6 and Delphi first.

Deliverables

First 6 to 12 months · everything public

Code under a permissive open-source licence, data and report under an open licence. Links are published here as each component is released.

Selected literature

Work this project builds on

1. Feathers, M. Working Effectively with Legacy Code. Prentice Hall, 2004. — Characterization testing: pinning down what a system does before changing it.
2. McKeeman, W. Differential Testing for Software. Digital Technical Journal, 10(1), 1998. — Comparing two implementations on identical inputs.
3. Lehman, M. Programs, Life Cycles, and Laws of Software Evolution. Proceedings of the IEEE, 68(9), 1980. — Why systems resist change as they age.
4. Chen, T.Y., Cheung, S.C., Yiu, S.M. Metamorphic Testing: A New Approach for Generating Next Test Cases. HKUST, 1998. — Testing without a known-correct oracle.
5. Klint, P., van der Storm, T., Vinju, J. RASCAL: A Domain Specific Language for Source Code Analysis and Manipulation. SCAM, 2009. — Dutch work on legacy analysis and transformation.
6. Rozière, B. et al. Unsupervised Translation of Programming Languages. NeurIPS, 2020. — Machine translation between programming languages.
7. Chen, M. et al. Evaluating Large Language Models Trained on Code. 2021. — Functional-correctness evaluation of generated code.
8. Pan, R. et al. Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code. ICSE, 2024. — Empirical failure modes in LLM code translation.
9. Jimenez, C. et al. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR, 2024. — Benchmark design for realistic software-engineering tasks.

References are to published work by others and imply no involvement of their authors in this project.

Context in the Netherlands

Related research, programmes and funding — no affiliation implied

Legacy modernisation in the Dutch public sector is studied and financed in several places. These organisations are listed because their work or funding scope touches this research question. Sofcom is independent of all of them and claims no partnership.

Research

• CWI — Software Analysis & Transformation

  National research institute for mathematics and computer science; long-running work on source-code analysis and the Rascal meta-programming language.

• TU Delft — Software Engineering Research Group

  Empirical software engineering, program analysis and automated testing.

• TNO

  Applied research organisation working on digital infrastructure for public bodies.

Public-sector programmes

• Common Ground (VNG)

  Municipal programme to decouple data from legacy applications — the policy context this research question sits in.

• ICTU

  Government foundation implementing ICT projects for public organisations.

• Digitale Overheid

  Government portal on digital-government policy, including legacy and lifecycle management.

Funding routes relevant to this work

• NWO

  Dutch Research Council; open competition and public-private calls in computer science.

• Regieorgaan SIA (RAAK)

  Funds practice-oriented research at universities of applied sciences — the natural route for a lectorate to join this project.

• NLnet Foundation / NGI Zero

  Grants for open-source work on an open, trustworthy internet; open deliverables are a requirement, which matches this project.

• SIDN Fonds

  Supports projects that strengthen the internet and digital autonomy in the Netherlands.

People

Project lead

Sofcom is a small, independent effort. Co-investigators are listed here by name as they join; this page states only people actually working on the project.

Collaborate

Open to researchers, lectorates and public-sector teams