A small library called go-simplequeue

Posted by Martijn on Wed, Mar 15, 2023

Go-simplequeue

Since I had need for a simple queueing system and it seemed like a fun project to quickly hack out, I built it.

Its a simple locking queue system with workers, written in Go. It served my needs, maybe it can serve yours.

I’m making it available as an Open Source library as I stongly believe in FOSS and giving back to the community.

Example of usage

 1// Define a job that conforms to the simplequeue.Job interface
 2type MyJob struct {
 3    id int
 4}
 5
 6func (mj *MyJob) ID() int64 {
 7    return int64(mj.id)
 8}
 9
10func (mj *MyJob) Do() {
11    // Lets just pause the job for a little time
12    ms := time.Duration(rand.Intn(1000)+1) * time.Millisecond
13    time.Sleep(ms)
14    fmt.Printf("Job %d executing\n", mj.ID())
15}
16
17// Create some jobs for our test
18func createJobs(number int) []*MyJob {
19    jobs := make([]*MyJob, 0)
20
21    for i := 1; i <= number; i++ {
22        jobs = append(jobs, &MyJob{id: i})
23    }
24
25    return jobs
26}
27
28// Run our program
29func main() {
30    ctx := context.Background()
31
32    // How much we want of each
33    numWorkers := 15
34    numJobs := 200
35
36    // Create some jobs with a helper function
37    jobs := createJobs(numJobs)
38
39    // Create a queue
40    q := sq.CreateQueue(ctx)
41
42    // Initialize the workers
43    workers := sq.InitializeWorkers(ctx, numWorkers)
44
45    fmt.Printf("Number of workers in pool: %d\n", len(workers))
46    fmt.Printf("Number of jobs for queue: %d\n", len(jobs))
47
48    // Push the jobs onto the Queue
49    for _, job := range jobs {
50        q.Push(job)
51    }
52
53    // Process the queue with some workers
54    q.Process(ctx, workers)
55
56    // Show some stats afterwards
57    var totalJobsHandled int64 = 0
58    for _, w := range workers {
59        totalJobsHandled += w.Handled()
60
61        fmt.Printf("Worker %d processed a total of %d jobs\n", w.ID(), w.Handled())
62    }
63
64    fmt.Printf("Total jobs handled: %d\n", totalJobsHandled)
65    fmt.Printf("Total workers: %d\n", len(workers))
66}

Licensing

Always a difficult topic. The go-simplequeue is made available under the MPL-2.0 license.

For more information on what that means in detail, you can read the license in full here: https://www.mozilla.org/en-US/MPL/

A slightly easier to read format can be found at https://choosealicense.com/licenses/mpl-2.0/

In essence: I chose this license because I want people that modify the code to give back to the community. The MPL-2.0 license makes sure that happens, protects me against patent trolls and still allows people to use the library as part of their own larger work & distribute that larger work with a license of their choice.

*) Disclaimer - I’m not a lawyer and this is not advice.