4.1 GitHub Integration and Git Source Control for Machine Learning Projects

GitHub Integration and Git Source Control for Machine Learning Projects

Quick Answer: Put the project in Git. Clone it onto a compute instance, GitHub Codespaces, or a laptop. Authenticate GitHub with a GitHub App (preferred) or a scoped personal access token (PAT) / SSH key — never a secret in the tree. When you submit a SDK/CLI v2 job from a clone, Azure Machine Learning stores the repo URI, branch, commit, and dirty flag on the job.

Domain 1 of Exam AI-300 clusters two official bullets: configure GitHub integration and manage source control with Git. The exam is not asking you to memorize git rebase trivia. It is asking you to run Machine Learning Operations (MLOps) so the same commit that a reviewer approved is the commit that trained the model, built the environment, and later provisioned the workspace.

Why Git is the source of truth

Azure Machine Learning fully supports Git for tracking work. You can clone onto the workspace file system, work on a local workstation, or run Git from a continuous integration / continuous delivery (CI/CD) pipeline. The platform does not require GitHub specifically: GitLab, Bitbucket, and Azure Repos are valid remotes. Exam AI-300 names GitHub because that is the automation host in the skills measured list (GitHub Actions appears in the next two sections).

A notebook that lives only in Studio Files is not source control. Two people edit the same train.py, nobody can tell which commit scored 0.91, and a deleted compute instance takes the only copy. Git fixes identity: a commit hash is the durable name of the code, the way a data asset version is the durable name of the data (Chapter 3).

When you submit a training job with the Python SDK v2 or Azure CLI ml extension v2, the client uploads the files the job needs. If the git binary is on the PATH of the submitter (your laptop, a compute instance terminal, or a GitHub-hosted runner) and those files sit inside a clone, the upload attaches Git metadata to the job. It is not tied to one central host — the values come from the local repo.

Read them in Studio (Jobs → job Overview → Raw JSON), from job.properties["mlflow.source.git.commit"] in SDK v2, or with az ml job show --query. If Git properties are missing, git --version failed or the files were outside the clone. Uncommitted edits (dirty: True) are a reproducibility incident, not a warning you ignore in production.

Recommended repository layout

Microsoft’s azureml-examples and Azure MLOps (v2) solution accelerator keep infrastructure, Azure Machine Learning YAML, and Python in one repo so a pull request can review all three. A layout that maps cleanly onto SDK/CLI v2 is:

Keep these out of Git (and out of GitHub Actions logs):

1. Secrets — PATs, AZURE_CLIENT_SECRET, storage account keys, shared access signatures, Key Vault connection strings, .env files with production values.
2. Credential JSON from az ad sp create-for-rbac --json-auth (deprecated, and still a secret if you generate it).
3. Default-datastore dumps and large model binaries — register them as data or model assets; do not git add a 4 GB checkpoint.
4. Notebook outputs with customer data. Strip outputs before you commit, or keep research notebooks on an experiment branch that never merges into main without review.

.gitignore for Python (__pycache__/, .venv/, .ipynb_checkpoints/) plus .env and *.pem is mandatory hygiene, not optional style.

Where you clone: Studio, compute instance, Codespaces

Azure Machine Learning gives every workspace a shared file system. The documented way to clone into it is: create a compute instance, open a terminal, and use the full Git client. You may also clone on a laptop and submit jobs with the SDK, or open the repo in GitHub Codespaces / Visual Studio Code and attach the Azure Machine Learning extension to a remote compute instance (Studio also has a VS Code remote flow).

Clone into your user directory so other workspace users do not checkout your experiment branch on top of you. HTTPS clone from a compute instance terminal uses a PAT. SSH clone uses a key you generate on the instance:

ssh-keygen -t ed25519 -C "you@contoso.com"
cat ~/.ssh/id_ed25519.pub

Add the public key to GitHub (or GitLab / Azure Repos / Bitbucket), then git clone git@github.com:org/repo.git. Put a passphrase on the key. The private key lives on that compute instance and is reachable only by the instance owner — still do not copy it into the repo or into a pipeline log.

GitHub Codespaces is the cloud editor GitHub hosts for the same repository. A Codespace is not an Azure Machine Learning compute target. You clone (GitHub already authenticated you to the repo), edit YAML and Python, and submit jobs with the Azure ML VS Code extension or the CLI after az login. Do not paste workspace keys into a Codespace devcontainer unless they come from GitHub encrypted secrets or Azure OpenID Connect (OIDC) — the same rule as Actions.

GitHub App versus PAT versus SSH

Three ways show up in Azure Machine Learning documentation and in GitHub’s own hardening guidance. Treat them as different jobs.

• SSH key on a compute instance — interactive humans cloning and pushing from that box. Microsoft’s Git integration article walks through ed25519 keys. Good for a single data scientist’s instance. Bad as the only org-wide automation identity.
• Personal access token (HTTPS) — what the compute-instance HTTPS path and the Git connection YAML use today. A CLI v2 Git connection looks like type: git, target: https://github.com/org/repo, and credentials.type: pat. Public repos omit credentials. Store the PAT in Azure Key Vault or a workspace connection, not in connection.yml committed to main.
• GitHub App — GitHub’s recommended automation identity. You install the App on the organization (or selected repositories). GitHub issues short-lived installation tokens. Permissions are repository-scoped (for example Contents: Read for clone-only, Contents: Read/Write if a bot must push a generated file). Org owners approve the install; you do not share a human password. Classic PATs are long-lived, often repo-wide on every repository the user can see, and survive after the engineer leaves if nobody rotates them.

If policy still requires a PAT, use a fine-grained PAT with an expiry, Contents limited to one repository, and no delete_repo. Rotate it. A leaked classic PAT in a notebook is a common exam trap and a real incident class.

Workspace Git connections (az ml connection create -f git.yml) are how Foundry/hub-style projects and some Studio clone flows hang a named credential on the workspace so you are not pasting a PAT into every terminal. They are not a substitute for GitHub branch protection.

Branching for experiments versus production

Keep the branching model boring:

1. main (or release/*) — production training pipelines, endpoint YAML, and infra/. Protected: required pull request, required reviewers, no force push.
2. dev or integration — merged feature work that may run against a dev workspace.
3. Feature / experiment branches — notebooks, hyperparameter sweeps, one-off data pulls. Jobs submitted from these branches still get a commit hash. They must not be what GitHub Actions deploys to the production workspace.

Do not “just train from main locally with uncommitted edits.” That sets azureml.git.dirty to True and means the job cannot be reproduced from the remote. Commit, push, open the pull request, then let Actions or a clean checkout submit the job.

Exam scenario

A claims team keeps train.py only on a compute instance. The instance is deleted to cut cost. Two weeks later a regulator asks which code produced model claims-seg-12. Studio shows metrics but no Git properties, and the fileshare clone never existed. The MLOps fix is: GitHub repo with src/, jobs/, environments/, and infra/; clone onto new instances from main; submit jobs from a clean checkout so mlflow.source.git.commit is on the job; store GitHub credentials as a GitHub App or a Key Vault–backed PAT, never in the notebook.

Common trap

Do not treat Studio Files or a compute-instance home directory as the backup. Local disk dies with the instance; the shared share is not a reviewable history. A sibling trap is putting a PAT in jobs/pipeline.yml or in a committed git-connection.yml. A third trap is assuming Azure Machine Learning “talks to GitHub” on every job — Git metadata is recorded only when the submitter has git and the files are inside a repository.