How to Automate Jenkins SCM Migration From Bitbucket to GitLab

Why This Even Matters

• Time sink: Clicking through job → Configure → Save for every pipeline is soul‑crushing.
• Risk of typos: One wrong character in an SCM URL silently breaks the build.
• Downtime: The longer the migration drags on, the longer your teams juggle two SCMs.

Automation = fewer clicks, fewer errors, faster cut‑over.

Prerequisites

• Python 3.6+
• Jenkins user with Configure permission + API token
• A list (or consistent pattern) of new GitLab URLs

Set your environment variables or drop them into a .env—Hard-coding creds in tutorials is for screenshots only.

Part 1 — Inventory Every Job

We start by crawling the folder tree, grabbing each job’s config.xml, and extracting the <url> from Git SCM blocks.

import requests, csv, xml.etree.ElementTree as ET
from urllib.parse import quote

JENKINS   = "https://jenkins.example.com"
USER      = "demo"
API_TOKEN = "••••••"
ROOT_PATH = "job/platform"  # top‑level folder to scan

auth = (USER, API_TOKEN)

def walk(folder: str):
    """Yield (name, full_path, xml_url) for every job under folder."""
    parts = "/job/".join(map(quote, folder.split("/job/")))
    url   = f"{JENKINS}/{parts}/api/json?tree=jobs[name,url,_class]"
    for j in requests.get(url, auth=auth).json().get('jobs', []):
        if 'folder' in j['_class'].lower():
            yield from walk(f"{folder}/job/{j['name']}")
        else:
            yield j['name'], f"{folder}/job/{j['name']}", j['url'] + 'config.xml'

def scm_url(xml_text: str):
    root = ET.fromstring(xml_text)
    x1   = root.find('.//hudson.plugins.git.UserRemoteConfig/url')
    x2   = root.find('.//source/remote')  # Multibranch
    return (x1 or x2).text if (x1 or x2) is not None else None

def main():
    rows = []
    for name, path, xml_url in walk(ROOT_PATH):
        xml = requests.get(xml_url, auth=auth).text
        url = scm_url(xml)
        if url:
            rows.append([name, path, xml_url[:-10], url])
            print(f"✓ {name}: {url}")

    with open('jenkins_scm_urls.csv', 'w', newline='') as f:
        csv.writer(f).writerows([
            ["Job", "Full Path", "Jenkins URL", "SCM URL"], *rows
        ])
    print(f"Exported {len(rows)} jobs → jenkins_scm_urls.csv")

if __name__ == '__main__':
    main()

You’ll walk away with a CSV you can slice and dice in Excel or awk.

Part 2 — Build a Mapping Sheet

Create replace.csv with Jenkins URL, Old SCM URL, and New SCM URL. Pattern fans can auto‑generate this with a one‑liner:

csvcut -c3,4 jenkins_scm_urls.csv \
  | sed 's#https://old-scm.com#https://gitlab.com/org#' > replace.csv

Part 3 — Bulk‑Update the Jobs

import requests, csv, xml.etree.ElementTree as ET, base64, time

JENKINS   = "https://jenkins.example.com"
USER      = "demo"
API_TOKEN = "••••••"
HEADERS = {
    'Authorization': 'Basic ' + base64.b64encode(f"{USER}:{API_TOKEN}".encode()).decode(),
    'Content-Type': 'application/xml'
}

def pull(url):
    return requests.get(url + 'config.xml', headers=HEADERS).text

def push(url, xml):
    return requests.post(url + 'config.xml', headers=HEADERS, data=xml).ok

def swap(xml, old, new):
    root, changed = ET.fromstring(xml), False
    for tag in ['.//hudson.plugins.git.UserRemoteConfig/url', './/source/remote']:
        for node in root.findall(tag):
            if node.text == old:
                node.text, changed = new, True
    return ET.tostring(root, encoding='utf‑8').decode() if changed else None

def update(row):
    url, old, new = row['Jenkins URL'], row['Old SCM URL'], row['New SCM URL']
    xml = pull(url)
    new_xml = swap(xml, old, new)
    return push(url, new_xml) if new_xml else False

def main():
    ok = fail = skip = 0
    with open('replace.csv') as f:
        reader = csv.DictReader(f)
        for row in reader:
            if update(row):
                ok += 1;  print('✓', row['Jenkins URL'])
            else:
                fail += 1; print('✗', row['Jenkins URL'])
            time.sleep(1)  # be kind to Jenkins
    print(f"Done: {ok} updated, {fail} failed, {skip} skipped")

if __name__ == '__main__':
    main()

Safety Checks Before You Hit Enter

1. Back up first: $JENKINS_URL/jenkins/script → println(Jenkins.instance.getAllItems()) isn’t a backup. Use the thin backup plugin or copy $JENKINS_HOME.
2. Run in dry‑run mode: Comment out push() and inspect the diff.
3. Throttle requests: Large shops may prefer a 5‑second delay or batch runs overnight.

What Could Possibly Go Wrong?

• Credential mismatch: New GitLab repo permissions must match Jenkins creds.
• Branch naming: If you renamed main/master, update your pipelines.
• Plugin quirks: Some multibranch jobs stash SCM URLs in additional nodes—grep for <remote> just in case.