Versioning large files can be problematic with distributed version control systems like Git. Git Large File Storage (or LFS) is a new, open-source extension to Git that aims to improve handling of large files.
It does this by replacing large files in your repository—such as graphics and videos—with simple text pointers. These pointers reference the large files, which are hosted elsewhere, either by GitHub or another external source such as an AWS bucket.
Initially this is confusing, but hopefully LFS will make more sense by the end of this tutorial.
Availability
LFS will be available to all GitHub users soon, but as I’m handling a lot of large files through GitHub with my Chip Shop board game, I signed up to the early access program to find out more.
Note: Unless you have Git LFS enabled on your account, this tutorial won’t work for you yet, but you can get an idea of the future feature.
Installing
If you have a Linux- or Windows-based system, visit git-lfs.github.com, download the installer, unarchive it and run the installer script.
If you have a Mac, do the same, but Git LFS is also available via Homebrew: install with brew install git-lfs.
Getting Started
Note: Git LFS will currently only work when using Git on the command line. I normally use Tower for managing my Git workflow, and this broke any LFS-related actions.
Create a Git repository as you normally do, and initialize the files you wish to track with LFS by issuing commands such as:
git lfs track "*.psd"git lfs track "*.mp3"Then use git as normal:
git add *.psd
git commit -m "Added PhotoShop files"
git push origin masterSo far, not a lot is different, but if we look into the details of a file on GitHub, we can see a subtle difference. Here’s a file hosted on GitHub:
And here’s a file hosted externally via LFS:
In the first image (a traditional GitHub repository), the file is located in the repository. In the second image (an LFS-enabled repository) the file is located in an AWS bucket.
Now go ahead and create some branches, make file changes, commit them and push:
Hang on, that .git folder is still large: wasn’t LFS supposed to handle files better?
This is where LFS gets somewhat confusing and possibly not as useful as you may have hoped.
To see what’s happening more clearly, delete the repo folder you created and then re-clone it from GitHub. You should see something like the below, with an appropriately sized .git folder:
Those files are ridiculously small! Try opening one, and you’ll likely see this message:
This is one of the aforementioned file references, and if you feel like cracking it open in a text editor you’ll see something like this:
version https://git-lfs.github.com/spec/v1
oid sha256:128b446a2cd06dd3b4dc2e2fe3336426792425870c3ada44ae7684b8391dc04d
size 1036867This is great if you’re a developer on a team, as you probably don’t need lots of media files cluttering your computer. And when it comes to deployment or testing, you likely have a build process that will assemble a project with real media. But what if you’re a designer and need to make changes? How do you access the real file?
Let’s see what commands are available to LFS by typing git lfs:
There are several that may be worth investigating in the future, but of most interest to us right now is fetch. By default, this will retrieve local copies of all files in the current checked out branch. This can be made specific by supplementing it with branches or commits:
# Fetches All
git lfs fetch# Fetches a particular branch
git checkout -b new_image
git lfs fetch new_imagegit lfs fetch 5a5c0ef0de779c9d4585320eab8d4a1bec696005And the files are available locally:
I would love a specific command to then remove the local file and replace it with a reference. Maybe there is another way of accomplishing this, or it will come in a future version.
Conclusion
Git LFS is a promising start, and I can see glimpses of genuine usefulness in the future. It needs better documentation and proper integration with 3rd party tools (the GitHub website included), which I’m sure are coming. If you have a decent Git, continuous integration and deployment system in place, then LFS will make far more sense. If you’re a small team, then it may be more of a bottleneck.
What are your thoughts? Useful or unnecessary?
Frequently Asked Questions (FAQs) about Git Large File Storage (LFS)
What is the main purpose of Git Large File Storage (LFS)?
Git Large File Storage (LFS) is an open-source Git extension for versioning large files. It replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub or GitLab. This means that you can work with large files in the same way as you would with other text files, but with much less strain on your system.
How does Git LFS differ from regular Git?
The main difference between Git LFS and regular Git lies in how they handle large files. Regular Git tracks changes in each file, which can be inefficient and slow for large files. Git LFS, on the other hand, stores a pointer to the large file in the repository, while the actual file is stored on a separate server. This makes Git LFS more efficient and faster when dealing with large files.
How do I install Git LFS?
To install Git LFS, you first need to download the software from the official Git LFS website. Once downloaded, open your terminal or command prompt and navigate to the directory where you downloaded Git LFS. Then, run the command ‘git lfs install’ to install Git LFS on your system.
How do I track large files with Git LFS?
To track large files with Git LFS, you use the ‘git lfs track’ command followed by the name of the file or files you want to track. For example, ‘git lfs track “*.psd”‘ would track all Photoshop files in your repository. Once you’ve done this, commit and push your changes as you normally would.
Can I use Git LFS with any Git repository?
Yes, you can use Git LFS with any Git repository. However, the remote repository must support Git LFS. Most popular Git hosting services, like GitHub and GitLab, support Git LFS.
What happens if I clone a repository with large files tracked by Git LFS?
When you clone a repository with large files tracked by Git LFS, Git LFS downloads the large files as needed, rather than downloading all the large files at once. This makes the cloning process faster and uses less disk space.
Can I stop tracking large files with Git LFS?
Yes, you can stop tracking large files with Git LFS by using the ‘git lfs untrack’ command followed by the name of the file or files you want to stop tracking. Once you’ve done this, commit and push your changes as you normally would.
How do I view the files tracked by Git LFS?
You can view the files tracked by Git LFS by using the ‘git lfs ls-files’ command. This will list all the files currently being tracked by Git LFS in your repository.
Can I use Git LFS with a private repository?
Yes, you can use Git LFS with a private repository. However, keep in mind that the large files tracked by Git LFS are stored on a separate server, so you’ll need to ensure that this server is also private and secure.
What are the limitations of Git LFS?
While Git LFS is a powerful tool for versioning large files, it does have some limitations. For example, it requires a separate server to store the large files, which can increase costs. Additionally, not all Git hosting services support Git LFS, so you’ll need to ensure that your chosen service does before you start using Git LFS.
Chris WardDeveloper Relations, Technical Writing and Editing, (Board) Game Design, Education, Explanation and always more to come. English/Australian living in Berlin, Herzlich Willkommen!
