Use Git LFS - Alibaba Cloud DevOps - Alibaba Cloud Documentation Center

This topic explains how to use Git Large File Storage (LFS) with Alibaba Cloud DevOps Codeup. It covers installation, tracking configuration, common operations, the underlying architecture, and known limitations.

Why choose Git LFS?

Git is designed to track changes in text-based source code. Every time a binary file such as an image, video, dataset, or compiled artifact is modified, Git stores a complete copy of the new version. Over time, this causes the repository to grow rapidly, which slows down cloning, fetching, and pulling for every contributor, even those who never use the binary files.

Git LFS solves this problem by storing large files on a separate server and replacing them in the repository with lightweight pointer files. It provides five major benefits:

Larger -- Supports version control for files in the gigabyte (GB) range.
Smaller -- Reduces the storage space used by Git repositories.
Faster -- Enables quicker cloning and pulling of repositories.
Transparent -- Standard Git commands work as expected. Git LFS usage is completely transparent to the user.
Compatible -- Fully compatible with permission control (compatible with Codeup permission control).

How Git LFS works

Before diving into setup and usage, it helps to understand how Git LFS operates at a high level.

Without Git LFS

Without Git LFS, all resources are fully stored on the Git server regardless of size. As the number of binary files grows and modifications become more frequent, the repository size expands quickly.

With Git LFS

With Git LFS, large files such as JPG images are uploaded to a dedicated large file storage service during the push process. Only the pointer files that reference these large files are pushed to the remote Git repository alongside regular code files.

Processing flow

When you add a tracked file, Git LFS replaces it with a pointer file in the Git object database and stores the actual content in the LFS cache. During a push, the large file content is uploaded to the LFS server separately from the Git objects.

Download and install Git LFS

Download

Choose the download method that matches your operating system:

Platform	Method
Linux (Debian and RPM)	Download packages from https://packagecloud.io/github/git-lfs/install
macOS	Run `brew install git-lfs`
Windows	Git LFS is integrated into Git for Windows. Install the latest version to get Git LFS included.
Binary package (all platforms)	Download from https://github.com/git-lfs/git-lfs/releases
Build from source	See https://github.com/git-lfs/git-lfs

Install

If you downloaded the binary package, extract it and run the installation script:

./install.sh

This script performs two tasks:

Installs the Git LFS binary executable in $PATH.
Runs git lfs install to enable the global LFS configuration in your environment.

After the script finishes, verify that Git LFS is initialized:

$ git lfs install
Updated pre-push hook.
Git LFS initialized.

Set up Git LFS for a new repository

The following walkthrough demonstrates a common scenario on Linux or macOS: creating a new repository on Codeup, configuring Git LFS, adding a large file, and pushing it to the remote.

Step 1: Create a new empty Git repository

Create a new empty repository named git-lfs on Codeup.

Clone the repository to your local machine and navigate to the directory:

   # Replace with your repository address
   $ git clone https://codeup.aliyun.com/your-organization-group/git-lfs.git
   Cloning into 'git-lfs'...
   warning: You appear to have cloned an empty repository.

   $ cd git-lfs

   $ tree .git/hooks/
   .git/hooks/
   ├── applypatch-msg.sample
   ├── commit-msg.sample
   ├── execute-commands.sample
   ├── post-receive.sample
   ├── post-update.sample
   ├── pre-applypatch.sample
   ├── pre-commit.sample
   ├── prepare-commit-msg.sample
   ├── pre-push.sample
   ├── pre-rebase.sample
   ├── pre-receive.sample
   └── update.sample

   0 directories, 12 files
   # At this stage, Git LFS related hooks have not been replaced

Step 2: Configure Git LFS tracking

Tell Git LFS to track files with the .bigfile extension:
Important
The double quotes ("") around *.bigfile are essential. Without them, your shell may expand the glob pattern before Git LFS can process it, which prevents the pattern from being recorded correctly. You can track files of any extension in the same way. For example, to track JPEG images, run git lfs track "*.jpg".
```
   $ git lfs track "*.bigfile"
   Tracking "*.bigfile"
```

Verify the currently tracked patterns:

   $ git lfs track
   Listing tracked patterns
       *.bigfile (.gitattributes)
   Listing excluded patterns

The track command writes the pattern to the .gitattributes file in the repository root. Add this file to the staging area:
```
   $ git add .gitattributes
```

Inspect the staged changes to confirm the tracking rule: The .gitattributes entry tells Git to use the LFS filter for diff, merge, and storage operations on all .bigfile files.

   $ git diff --cached
   diff --git a/.gitattributes b/.gitattributes
   new file mode 100644
   index 0000000..c441ad2
   --- /dev/null
   +++ b/.gitattributes
   @@ -0,0 +1 @@
   +*.bigfile filter=lfs diff=lfs merge=lfs -text

Step 3: Activate the Git LFS configuration

Commit the .gitattributes file to activate the LFS tracking configuration for *.bigfile:

$ git commit -m "Add \"*.bigfile\" LFS config "
[master (root-commit) d052478] Add "*.bigfile" LFS config
 1 file changed, 1 insertion(+)
 create mode 100644 .gitattributes

$ git log --oneline
d052478 (HEAD -> master) Add "*.bigfile" LFS config

Step 4: Create a large file for testing

Create a 1 GB test file named dyrone.bigfile in the working directory:

# On macOS, you can use the mkfile command instead of dd
$ dd if=/dev/zero of=dyrone.bigfile bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.41392 s, 445 MB/s

$ du -sh dyrone.bigfile
1.1G    dyrone.bigfile

Add the file to the staging area:

$ git add dyrone.bigfile

Because the file extension .bigfile matches the pattern *.bigfile in .gitattributes, Git LFS intercepts the file and processes it as an LFS object instead of a regular Git blob.

Step 5: Push the file to the remote

Commit the file and push to the remote repository:

$ git commit -m "Add a really big file"
[master 8032589] Add a really big file
 1 file changed, 3 insertions(+)
 create mode 100644 dyrone.bigfile

The output 1 file changed, 3 insertions(+) indicates that the pointer file (which has 3 lines) was committed, not the full 1 GB file. You can verify this by viewing the commit details:

$ git show HEAD
commit 8032589f47a748171e84da94ce6440fe139e99f9 (HEAD -> master)
Author: dyroneteng
Date:   Tue Sep 15 17:25:58 2020 +0800

    Add a really big file

diff --git a/dyrone.bigfile b/dyrone.bigfile
new file mode 100644
index 0000000..9d7c19f
--- /dev/null
+++ b/dyrone.bigfile
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49bc20df15e412a64472421e13fe86ff1c5165e18b2afccf160d4dc19fe68a14
+size 1073741824

Now push the commit to the Codeup repository:

$ git push
Uploading LFS objects:   0% (0/1), 3.9 MB | 0 B/s
Uploading LFS objects:   0% (0/1), 79 MB | 30 MB/s
Uploading LFS objects:   0% (0/1), 207 MB | 50 MB/s
Uploading LFS objects:   0% (0/1), 326 MB | 51 MB/s
Uploading LFS objects:   0% (0/1), 534 MB | 56 MB/s
Uploading LFS objects: 100% (1/1), 1.1 GB | 58 MB/s, done.
Counting objects: 3, done.
Delta compression using up to 32 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 410 bytes | 410.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://codeup.aliyun.com/xxxxx/git-lfs.git
   d052478..8032589  master -> master

At this point, all .bigfile files in this repository are managed by Git LFS, while other files continue to be managed by regular Git.

Note

When a push includes Git LFS files, the process is divided into two stages: 1. Upload LFS objects: Git LFS uploads the actual large file (identified by its oid) to the LFS server. The line Uploading LFS objects: 100% (1/1), 1.1 GB | 58 MB/s, done. shows the upload progress, including the number of files, total size, and transfer speed. 2. Push Git objects: After the LFS upload completes, Git pushes the pointer file and related objects (blob, tree, commit) to the Git server using the standard Git protocol. The line Writing objects: 100% (3/3), 410 bytes | 410.00 KiB/s, done. confirms the three objects were pushed. If either stage fails, the entire push is unsuccessful.

The key benefit of this approach is visible in the repository size. The local repository (excluding the LFS cache directory) is only about 188K, roughly the same size as the remote Git repository. The LFS cache directory holds the actual large files separately:

$ du -sh --exclude=lfs .git
188K    .git

$ du -sh .git
1.1G    .git

Pointer files and local LFS storage

This section explains the internal mechanisms that Git LFS uses when you stage and store tracked files.

What is a pointer file?

When you stage an LFS-tracked file, Git LFS does not write the full file content into the Git index (staging area). Instead, it stores a small pointer file that references the actual LFS object. You can inspect the pointer file by running git diff --cached:

$ git diff --cached
diff --git a/dyrone.bigfile b/dyrone.bigfile
new file mode 100644
index 0000000..9d7c19f
--- /dev/null
+++ b/dyrone.bigfile
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49bc20df15e412a64472421e13fe86ff1c5165e18b2afccf160d4dc19fe68a14
+size 1073741824

The pointer file contains three fields:

Field	Description
`version https://git-lfs.github.com/spec/v1`	The Git LFS protocol version. `git-lfs.github.com` is the official address of the Git LFS open-source project.
`oid sha256:<hash>`	The object ID. A 64-character hexadecimal value computed by using SHA-256, which serves as a unique identifier for the file content.
`size`	The actual file size in bytes.

How are local LFS files stored?

After the pointer file is staged, the actual file (for example, dyrone.bigfile at 1 GB) is stored in the LFS cache directory of the repository. The file name in the cache corresponds to the oid value from the pointer file:

$ tree .git/lfs
.git/lfs
├── objects
│    └── 49
│         └── bc
│             └── 49bc20df15e412a64472421e13fe86ff1c5165e18b2afccf160d4dc19fe68a14
└── tmp
4 directories, 1 file

In the local repository, Git LFS stores large files as two parts: the blob of the pointer file is the object referenced by the commit, while the actual LFS file is cached in the .git/lfs directory. The file in the working directory remains unchanged and usable as-is.

Clone a repository that uses Git LFS

Important

Git LFS must be installed on your machine before you clone a repository that uses Git LFS. If Git LFS is not installed, the LFS-tracked files in the cloned repository will appear as pointer files instead of the original files.

When you clone a repository that uses Git LFS, git-lfs automatically replaces the relevant hooks in the repository to enable LFS functionality:

# Sample repository. Replace with the actual address of your LFS-enabled repository.
$ git clone git@codeup.aliyun.com:your-organization-group/dyrone.git
Cloning into 'dyrone'...
remote: Counting objects: 41, done.
remote: Total 41 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (41/41), 2.98 MiB | 5.01 MiB/s, done.
Filtering content: 100% (3/3), 5.01 MiB | 3.83 MiB/s, done.
# "Filtering content" indicates that the downloaded LFS pointer files
# are being replaced with the actual files in the working directory.

$ cd dyrone

$ tree .git/hooks
.git/hooks
├── applypatch-msg.sample
├── commit-msg.sample
├── execute-commands.sample
├── fsmonitor-watchman.sample
├── post-checkout
├── post-commit
├── post-merge
├── post-update.sample
├── pre-applypatch.sample
├── pre-commit.sample
├── pre-merge-commit.sample
├── pre-push
├── pre-push.sample
├── pre-rebase.sample
├── pre-receive.sample
├── prepare-commit-msg.sample
└── update.sample

0 directories, 17 files

Notice that the Git LFS hooks (post-checkout, post-commit, post-merge, pre-push) are now active (they appear without the .sample suffix). These hooks ensure that Git LFS operates transparently during standard Git operations such as checkout, commit, merge, and push.

Migrate historical files to Git LFS

If your repository already contains large binary files that are tracked by regular Git and you want to migrate them to Git LFS, refer to the LFS Migration Guide for detailed instructions on handling complex migration scenarios.

Untrack files from Git LFS

To stop tracking a file type with Git LFS and return it to regular Git management, follow these steps:

Remove the tracking rule and clean the files from the LFS cache:

   git lfs untrack "*.bigfile"
   git rm --cached "*.bigfile"

Add the files back to regular Git tracking and commit the change:

   git add "*.bigfile"
   git commit -m "restore *.bigfile to git from lfs"

Limitations

Windows 4 GB file size limit: On the Windows platform, a single LFS file is limited to a maximum size of 4 GB. For details, see issue 2434.
Windows Git Credential Manager required: Windows users must ensure that Git Credential Manager is installed. Otherwise, LFS operations may hang indefinitely. For details, see issue 1763.
LFS download token timeout: Unlike GitLab, which uses a hardcoded LFS download token timeout of 30 minutes, Codeup dynamically calculates the token timeout based on the list of files to be downloaded. However, in environments with poor network conditions, token timeouts may still occur. If you need to adjust the timeout, contact the Codeup system administrator for assistance.