Designing and building LeetCode - Part II

Introduction

Around September 2024 I built a LeetCode alternative. My plan was to create better, more practical problems/challenges and launch it as a small product.

It had a promising start, but eventually I shut it down because it had 0 paying customers, and the system was a bit too complex to maintain it as a free app.

At least now, I can share how it works because it has some interesting stuff:

• Remote code execution

• Docker container pooling

• Asserting user submissions

In part I we created a basic container pool and we wrote the Get method:

func (p *ContainerPool) Get(language *language.Language) (string, error) {
  p.mu.Lock()
  defer p.mu.Unlock()

  containers, ok := p.containers[language.ID]
  if !ok || len(containers) == 0 {
    return p.createContainer(language)
  }

  id := containers[len(containers)-1]
  p.containers[language.ID] = containers[:len(containers)-1]

  res, err := p.cli.ContainerInspect(context.Background(), id)
  if err != nil {
    sentry.CaptureException(
      fmt.Errorf("trying to get stopped container: %s: %w", id, err)
    )
    return p.createContainer(language)
  }
  if !res.State.Running {
    sentry.CaptureException(
      fmt.Errorf("trying to get stopped container: %s: %w", id, err)
    )
    return p.createContainer(language)
  }

  return id, nil
}

In this part, we’re going to continue this by exploring how to create containers and how to push containers back to the pool.

Get the book

This is going to be a 3-part series in the next 3 weeks.

I also put together a 72-page book, that contains more knowledge and the whole 13k line codebase to the project.

• High-level design

• Docker inside Docker

• Container pools

• Code execution

• Asserting test cases

• Init and tear down tasks for containers

• Package oriented design

• How to navigate the source code

Creating a container

The createContainer function is an important part of Get. If there are no available containers in the pool it needs to create a new one on-the-fly.

The Docker CLI provides two function to create and start containers:

ContainerCreate()
ContainerStart()

ContainerCreate takes two configs:

• A config for the container

• Another one for host configurationsú

The container config contains stuff like these:

{
  "image": "php:8.3.0",
  "env": ["FOO=foo","BAR=bar"],
  "healtcheck": {...}
}

It contians configs that you would use when starting a contianer using your terminal. But these are specific to the container. Anything related to the host machine goes into the host config:

{
  "restart_policy": {
    "name": "unless-stopeed"
  },
  "port_bindings": {...},
  "mounts": {...}
}

It’s clear that we need to apply different configs depending on the submission's language. If a user submits a solution in Golang we need config like this:

{
  "image": "go:1.23",
  "healtcheck": {...}
}

And so on.

In my opinion, the simplest solution is to create a config for each language and store them in some kind of store. A config looks like this:

{
  "execute_cmd": ["php"],
  "healtcheck_cmd": ["CMD", "php", "-v"],
  "image_url": "docker.io/library/php:8.3.11-cli-alpine3.20",
  "image": "php:8.3.11-cli-alpine3.20",
  "env": [],
}

ecxecute_cmd is the actual we want to execute when running the user’s code. The actual commands will look like this:

php solution.php
go run solution.go
node solution.js

These commands will be passed to a docker container. The bold part is the actual execute_cmd stored in the config.

healthcheck_cmd defines the health check for the given language. Since these containers live for a long time and will be re-used to run multiple submissions, it’s important to include health checks. A health check is executed by docker and if it fails the container will be restarted automatically.

For a simple container that only contains a programming language a simple version command is a good health check:

php -v
node -v
go version

LeetCode offers database-related challenges. My version also included MySQl and Redis related problems. Their health checks look like this:

mysql -uroot -proot -e show databases
redis-cli ping

The other fields in the config, such as, image_url, image, and env are self-explanatory.

To store these config I went with the most simple solution. It’s a hashmap. I don’t store these in a database, or file, not even Redis. Just a hashmap in the code:

var store = map[string]*Config{
  "php": {
    ExecuteCmd:     []string{"php"},
    InitCmd:        []string{"tail", "-f", "/dev/null"},
    HealthCheckCmd: []string{"CMD", "php", "-v"},
    ImageURL:       "docker.io/library/php:8.3.11-cli-alpine3.20",
    Image:          "php:8.3.11-cli-alpine3.20",
    Env:            nil,
  },
  // ...
}

And there’s a GetConfig function that returns a config for a specific language.

Once these configs are added, creating a container is pretty easy:

func (p *ContainerPool) createContainer(lang *language.Language) (string, error) {
  cnf, err := container.GetConfig(lang.ID)
  if err != nil {
    return "", err
  }

  containerCnf := &dockercontainer.Config{
    Env:   cnf.Env,
    Image: cnf.Image,
    Healthcheck: &dockercontainer.HealthConfig{
      Test:          cnf.HealthCheckCmd,
      Interval:      time.Second * 10,
      Timeout:       time.Second * 3,
      StartPeriod:   time.Minute * 2,
      StartInterval: time.Second * 10,
      Retries:       2,
    },
  }
}

We grab the right config based on the language and then the function transforms it into a Config object provided by Docker CLI.

It might be a bit confusing, but container is a package (like a namespace in other languages) in the app itself, and the Docker CLI lib also provider a package called container so I aliased it to dockercontainer.

After that, we can create the container:

res, err := p.cli.ContainerCreate(
  context.Background(),
  containerCnf,
  &dockercontainer.HostConfig{
    RestartPolicy: dockercontainer.RestartPolicy{
      Name: dockercontainer.RestartPolicyUnlessStopped,
    },
  },
  nil,
  nil,
  "",
)
if err != nil {
  return "", fmt.Errorf("unable to create container in pool: %w", err)
}

It receives the language-specific container config and a generic host config. It only contains the unless-stopped restart policy.

The result contains the container ID such as:

cc5564edfe81

With this ID the container can be started:

err = p.cli.ContainerStart(
  context.Background(), 
  res.ID, 
  container.StartOptions{},
); 
if err != nil {
  return "", fmt.Errorf("unable to start container in pool: %w", err)
}
return res.ID, nil

Since the container pool contains a hash map with lists inside and some other dependencies (such as docker CLI) we can add a constructor:

func NewContainerPool(
  capacity int, 
  cli *client.Client, 
  logger *zap.Logger, 
  cnf *config.Config
) *ContainerPool {
  containers := make(map[string][]string)
  for _, lang := range language.FindAll() {
    containers[lang.ID] = make([]string, 0)
  }

  return &ContainerPool{
    cli:        cli,
    capacity:   capacity,
    containers: containers,
    logger:     logger,
    config:     cnf,
  }
}

(“constsructors” in Go are just simple functions starting with the word “New”)

It initialized the hash map for with languages and sets the other fields as well.

Now we have a basic container pool that manages containers. It can be used like this:

func runSubmission(lang *language.Language) error {
  pool := NewContainerPool(3, cli, logger, cnf)

  id, err := pool.Get(lang)
  if err != nill {
    return err
  }
  defer pool.PushBack(lang, id)

  fmt.Printf("my container ID: %s", id)
}

It’s important to call PushBack after the container is no longer needed. For this purpose, defer can be used. It runs the given function when the current function (runSubmission) returns. It’s useful because if runSubmission is 50 lines long it’s very easy to forget about calling PushBack at the end.

The only “disadvantage” of using defer is error handling. If PushBack can fail (and it can) the proper defer looks like this:

func runSubmission(lang *language.Language) (e error) {
  pool := NewContainerPool(3, cli, logger, cnf)

  id, err := pool.Get(lang)
  if err != nill {
    return err
  }
  defer func(){
    if err = pool.PushBack(lang, id); err != nil {
      e = err
    }
  }()

  fmt.Printf("my container ID: %s", id)
}

Since the deferred function has no return value, you cannot just simply return the error return by PushBack. You actually need to have a named return value:

func runSubmission(lang *language.Language) (e error)

And the set e inside the deferred function if nedded:

if err = pool.PushBack(lang, id); err != nil {
  e = err
}

Populating the pool

As we discussed in part I, the container pool should be populated with pre-created containers when the application starts. It involves two steps:

• Pulling the required images

• Starting containers

We already know how to start containers so let’s pull images. For this, Docker CLI provides the following func:

ImagePull(context, url, options)

url is the full image URL such as docker.io/library/php:8.3.11-cli-alpine3.20

options is an object that most importantly contains the docker registry auth credentials in base64 encoded format.

If you remember, the ContainerPool struct contains a config field:

type ContainerPool struct {
  containers map[string][]string
  capacity   int
  config *config.Config
  // ...
}

This Config object is the application config and it contains the registry credentials.

The first part of the function looks like this:

func (p *ContainerPool) prePullImages(
  languages map[string]*language.Language
) error {
  cnf := registry.AuthConfig{
    Username: p.config.DockerHubUsername,
    Password: p.config.DockerHubToken,
  }
  jsonData, err := json.Marshal(cnf)
  if err != nil {
    return fmt.Errorf("unable marshal registry credentials: %w", err)
  }
  encoded := base64.StdEncoding.EncodeToString(jsonData)
  opts := image.PullOptions{
    RegistryAuth: encoded,
  }
}

In order to base64 encode the credentials, first we need to convert the object to JSON which is done json.Marshal().

Golang has an amazing stdlib, especially when it comes I/O and converting data. Marshal and Unmarshal are common in data transformation. They translate to these PHP functions:

json_encode() -> json.Marshal()
json_decode() -> json.Unmarshal()

Once we encoded the registry credentials we can pull an image for each language:

for _, lang := range languages {
  cnf, err := container.GetConfig(lang.ID)
  if err != nil {
    return err
  }

  r, err := p.cli.ImagePull(context.Background(), cnf.ImageURL, opts)
  if err != nil {
    return fmt.Errorf(
      "unable to preppull image: %s: %w", cnf.ImageURL, err
    )
  }

  if _, err = io.Copy(os.Stdout, r); err != nil {
    p.logger.Error("unable to copy output",
      zap.Any("err", err),
    )
  }
  if err = r.Close(); err != nil {
    p.logger.Error("unable to close reader",
      zap.Any("err", err),
    )
  }
}

As we discussed earlier, each language has a config that contains the required docker image URL.

ImagePull doesn’t return the actual output of the operation but an io.ReadCloser which is the combination of io.Reader and io.Closer. If you’re used to PHP, it seems pretty weird at first. I/O operations in PHP works in a “Write a string. Return a string.” fashion. In Go, there are Writers and Readers everywhere. However, this article isn’t about Golang so let’s not go into the details.

The important thing is that an io.Reader can be read and can be “piped” to an io.Writer. If we want to print out the results, we can do this:

io.Copy(os.Stdout, r)

io.Copy takes an io.Writer (stdout, in this case) and an io.Reader. It reads the reader and writes it to the writer in 32KB chunks. So it’s memory efficient and buffered by default.

If you would rather have 4KB chunks you can do this:

buf := make([]byte, 4*1024)
io.Copy(os.Stdout, r, buf)

After copying, the reader must be closed as almost all resources (files, TCP connections, etc).

So now we can create container and pull all the images. We have everything for populating the entire pool:

func (p *ContainerPool) Populate(
  languages map[string]*language.Language
) error {
  err := p.prePullImages(languages)
  if err != nil {
    return err
  }

  for _, lang := range languages {
    for i := 0; i < p.capacity; i++ {
      containerID, err := p.createContainer(lang)
      if err != nil {
        return err
      }

      err = p.PushBack(lang, containerID)
      if err != nil {        
        return err
      }
    }
  }

  return nil
}

After pulling the images, it loops through the languages and creates X containers where X is the capacity. After creating the container, the function pushes it into the pool.

In the real function there are some log entries and detailed error messages, but I simplified it so it’s easier to read.

Also, in a function like this you can choose to handle errors “silently”, for example:

for _, lang := range languages {
  for i := 0; i < p.capacity; i++ {
    containerID, err := p.createContainer(lang)
    if err != nil {
      p.logger.Error("...")
      continue
    }
    // ...
  }
}

Let’s say the system handles 5 languages. For each language, you want to create 5 containers. It’s 25 containers. If one cannot be created it’s not a big deal, right? So you can just log the error and move on. No big deal.

It sounds reasonable, but I have two problems with that:

• If the 6th container creation goes wrong and you skip it, you don’t know what will happen in the next 19 iterations. What if docker stopped working? You’ll end up with only 6 containers instead of 25. And your system won’t work, because docker is not responding.

• This function runs at every startup. It initializes basically the entire system. Everything else depends on this setup. I would not allow any error here.

So now the entire container pool is ready. It can:

• Retrieve existing containers

• Create new ones

• Pull images

• Populate the pool

The Populate function should be called in the main function (similar to index.php):

package main

func main() {
  pool, err := executor.NewContainerPool(5, cli, logger, cnf)
  if err != nil {
    log.Fatal(err)
  }

  err = pool.Populate(language.FindAll())
  if err != nil {
    log.Fatal(err)
  }
}

To be continued

In the next article, we’re going to execute user submitted code.

Get the book

This is going to be a 3-part series in the next 3 weeks.

I also put together a 72-page book, that contains more knowledge and the whole 13k line codebase to the project.

• High-level design

• Docker inside Docker

• Container pools

• Code execution

• Asserting test cases

• Init and tear down tasks for containers

• Package oriented design

• How to navigate the source code