How I used git submodule, sparse checkout and npm workspaces in a recent project!

2023-03-21 . 2 min read

Background

I've been obsessed with markdown ever since I was introduced to it. This blog post itself was initially written in markdown for easy formatting during editing so I get an early idea of how the post is finally gonna look. Also, I realized that I was really curious about how all this formatting, linting, and parsing actually works. Of course, I couldn't imagine the current me developing a fully functional code editor myself with the ability of parsing and linting the code, so I used the awesome codemirror package by the awesome Marijin Heverbeke. Honestly, at first, it was really hard using the package even though I used it before but once you understand the editor's system it's really easy and really really extensible.

Now the idea of using MD for taking notes or simply formatting is really popular and most of us have been using it in Notion and GitHub. I wanted to create such a mini version myself. So I developed a simple note taking app; I call it snote short for short note. The deployed version is really simple and uses indexed DB, so all your notes are stored locally.

Use Submodules

I already had a working MD Editor and here's what I wanted:

use the MDEditor package which is on GitHub but not hosted on npm
use it as a package and not as part of this project (snote)

Git Submodules to the rescue. And here's the funny thing; The intro line of the submodules page writes and I quote:

It often happens that while working on one project, you need to use another project from within it.

Using submodules is so easy; You literally have to run just one command on the terminal and you're set.

 git submodule add <PATH_TO_REPO> <OPTIONAL:LOCAL_PATH_TO_ADD_SUBMODULE>

Wow, so easy, what other problem can I have? I was set! or so I thought 😕. The repo containing the md-editor package has two main folders editor and web. The editor package lives in the editor folder and the web folder contains the website for testing the package. In fact, I am currently using the same site for testing the format of this blog post. So, out of the two main folders in the md-editor repo, I only wanted:

the editor as a package and not track the web from the snote repo.

Luckily, git has this feature called Sparse Checkout which is the exact solution to my problem.

But again I ran into another problem. I could not find a way to add a sparse submodule directly. So, what I did was, clone the editor package sparsely under the main project and then add the sub repo as a submodule. It worked!

Here are the step-by-step commands.

# Create a folder where you want the submodule
mkdir editor && cd editor

# Let git know that we are cloning sparsely; doesn't download any files yet
git clone --filter=blob:none --no-checkout https://github.com/Rounak-stha/markdown-editor.git .

# I don't know what it does
git sparse-checkout set --cone

# Checkouts only base files like package.json
git checkout main

# Clones Editor directory only
git sparse-checkout set Editor

# goto the root directory of the project
cd ..

# Add Submodule pointing to already existing sparsely cloned local repo
git submodule add https://github.com/Rounak-stha/markdown-editor.git editor/

# If the package has any new changes,
# Fetch New changes in SubModule
git submodule update --remote editor

Workspaces

Git knows about this submodule but the package manager doesn't. I was using npm for the project. Each package has its own dependencies which is mentioned in the package.json file at the root of each package. Just like when using npm packages normally, I wanted to just run npm install to download all the required dependencies and use one core node_modules folder. Package managers give us this feature called workspaces. With this command:

npm init -w ./editor

We let npm know about this package we are using, so it includes the dependencies of this workspace as well when we run npm i. Npm creates a woskspaces field on the root package.json file to track all the workspaces. And now we are set.

Links

Hey I assume you finished reading, I would love to know your feedback or if found any error or mistake in this blog post, please do not hesitate to reach out to me.