The use of GenAI to Migrate and Modernise Organisational Core Programming Languages

Share this post:

GenAI is hugely powerful and supports a diversity of use cases by focusing on routine work – allowing people to focus time on value-add tasks, thus enhancing productivity. The focus of this use case is for an organisation which had previously focussed on a legacy set of tooling and programming languages and needed a way to accelerate the migration to a more modern, business appropriate set of technologies. This will improve business productivity and deliver a faster time to impact by allowing the business to attract a wider pool of possible team members and collaborators.

This problem is deeply rooted in the organisation, huge amounts of time has been invested in developing the skills for the legacy applications and the code repository – core to the business and for individual productivity. Over time, transition becomes increasingly more difficult as the skills for the legacy toolset dwindle. For most companies this can be a daunting task; the transition can be complicated as systems and people are trained on new languages, and the new tooling is on-boarded.

Traditionally, an organisation would spend time creating an interpreter of the existing language and converting statements via some intermediate state, into the target language, taking time and effort with many retries. This process requires experienced personnel with in-depth knowledge of how to do the conversion including the source, target, and conversion languages.

With GenAI, this process can be achieved in tens of minutes to develop the required prompts compared to the days of effort using traditional approaches. The GenAI tools also provide a coach that gives an explanation including descriptions and summarisations of the translated code. This creates significant business benefits in aiding learning but requires significant development effort with traditional tooling.

Business Problem

The core of this use case is that Data scientists when they first join the client organisation are trained on a legacy statistical programming language. This language is hosted in an advanced statistical data science tool with its own proprietary programming language. Today hundreds of Data Scientists use that and other tools to perform their work and have built up a legacy of code and know-how. For a variety of reasons, the client has decided to migrate their staff from this statistical language to R, but has a legacy of thousands of files which need to be converted while the teams are learning R.

Gen AI can drive code conversion providing a rapid approach to modernising existing code assets at scale by providing a first draft output for refinement by the “human in the loop”. This has the potential to save significant client time to manually convert the existing statistical code files and provide modernisation of undocumented code.

What We Did

We obtained code samples consisting of different line lengths and complexity to be used in the evaluation.

Using IBM Watson X.AI hosted on the IBM Cloud, allows different Large Language Models (LLM’s) to be accessed from a single interface. LLM’s interact through Prompts; Prompts are the natural language (questions/ summarisation/ language translation/..) statements that are asked of the LLM to perform the operations that were needed.

After we had a basic working Prompt, we wrote a Python script to connect to the IBM Watson-X.Ai API and generate the required outputs. We initially chose an LLM using Meta LLama-2 70b, then moved to the Mistral instruct v0-2 model (IBM Announces Availability of Open-Source Mistral AI Model on watsonx) that could operate standalone and provide required conversion capability.

Two Prompts were developed (see picture above), the first performed the conversion of the statistical code to R. The LLM answers the prompt, with inbuilt knowledge to perform the translation once the prompt is correctly defined. The second prompt asked the LLM to generate a textual description of the code that was being converted; This was in form of a virtual author generating a narrative of the R code, in simple clear written English.

The LLM did not need to be trained for this task, the knowledge and understanding was already embedded in the model. This saves time and effort on behalf of the development community, although it is possible to extend the LLM capabilities.

LLMs are very powerful but have restrictions in terms of number of lines of text that can be processed at one time. The Python code split the statistical code into fixed number of lines, based on the last nearest comment or blank line, and then submitted to the LLM with the prompt, the results combined to create a complete output.

However, there are risks: LLMs can misinterpret the prompt or the source data and generate false statements, simply through not enough trained data or at an extreme through hallucination. Therefore, there must be thorough testing to ensure that the outputs of both code sets match.

To help automatically measure the LLM performance, there is an option to use IBM Watsonx.AI tooling has IBM Watsonx.governance which is designed to monitor artificial intelligence activities across an organisation, using GenAI or machine learning models from any vendor. The tool supports evaluation and monitoring of the model for health, accuracy, drift, bias and Gen AI quality.  It provides powerful governance, risk and compliance capabilities featuring workflows with approvals, customizable dashboards, risk scorecards and reports.  IBM watsonx.AI uses factsheet capabilities to collect and document model metadata automatically across the AI model lifecycle. However, we did not have time to implement it in this use case.

Through this process, care was taken to protect the client data so that it did not move outside of the environment. This means as stated earlier the LLM was locally hosted on the IBM cloud and the platform could not connect with the outside world as part of their operation. Some hosted LLMs required connection through an internet accessible API, which would not have been allowed.

Aware of, and mitigating the risks, it is easy to see conversion factories being setup where an end user can submit their code and ask for conversion into a specific language and by return get the output and description of the code that has been developed. A further enhancement could be that the GenAI builds replica systems in the cloud of choice; Using the code and other system artefacts as a template, delivering back the e.g. Python, Terraform for deployment where whole systems could be migrated in this way.

The Outcome

Based on feedback from the client, the completed work demonstrated that the R code was approx. 80% accurate and easily correctable to work as expected. Given more time, accuracy of the conversion could have been improved through further refinement of the Prompts. In addition, the English description was very readable and helped the understandability of the converted R code. The process takes minutes, compared to hours of translating by hand or traditional methods, but will still need a human (or possibly automated test mechanism) to verify the output.

Given the scale of the task, the business value in time saving and accuracy is significant.  A further benefit is showing the team side-by-side, the translation from a language they were familiar with, to the new language; This will be invaluable in developing  R skills and accelerating adoption in the teams.

Written by Mark Restall, Matt Sargeant, Joe Schuller, Russ Latham, Nicole Mather