Go-simplequeue
Since I had need for a simple queueing system and it seemed like a fun project to quickly hack out, I built it.
Its a simple locking queue system with workers, written in Go. It served my needs, maybe it can serve yours.
I’m making it available as an Open Source library as I stongly believe in FOSS and giving back to the community.
- Github: https://github.com/mvdkleijn/go-simplequeue
- Generated Go packages info: https://pkg.go.dev/github.com/mvdkleijn/go-simplequeue
Example of usage
1// Define a job that conforms to the simplequeue.Job interface
2type MyJob struct {
3 id int
4}
5
6func (mj *MyJob) ID() int64 {
7 return int64(mj.id)
8}
9
10func (mj *MyJob) Do() {
11 // Lets just pause the job for a little time
12 ms := time.Duration(rand.Intn(1000)+1) * time.Millisecond
13 time.Sleep(ms)
14 fmt.Printf("Job %d executing\n", mj.ID())
15}
16
17// Create some jobs for our test
18func createJobs(number int) []*MyJob {
19 jobs := make([]*MyJob, 0)
20
21 for i := 1; i <= number; i++ {
22 jobs = append(jobs, &MyJob{id: i})
23 }
24
25 return jobs
26}
27
28// Run our program
29func main() {
30 ctx := context.Background()
31
32 // How much we want of each
33 numWorkers := 15
34 numJobs := 200
35
36 // Create some jobs with a helper function
37 jobs := createJobs(numJobs)
38
39 // Create a queue
40 q := sq.CreateQueue(ctx)
41
42 // Initialize the workers
43 workers := sq.InitializeWorkers(ctx, numWorkers)
44
45 fmt.Printf("Number of workers in pool: %d\n", len(workers))
46 fmt.Printf("Number of jobs for queue: %d\n", len(jobs))
47
48 // Push the jobs onto the Queue
49 for _, job := range jobs {
50 q.Push(job)
51 }
52
53 // Process the queue with some workers
54 q.Process(ctx, workers)
55
56 // Show some stats afterwards
57 var totalJobsHandled int64 = 0
58 for _, w := range workers {
59 totalJobsHandled += w.Handled()
60
61 fmt.Printf("Worker %d processed a total of %d jobs\n", w.ID(), w.Handled())
62 }
63
64 fmt.Printf("Total jobs handled: %d\n", totalJobsHandled)
65 fmt.Printf("Total workers: %d\n", len(workers))
66}
Licensing
Always a difficult topic. The go-simplequeue is made available under the MPL-2.0 license.
For more information on what that means in detail, you can read the license in full here: https://www.mozilla.org/en-US/MPL/
A slightly easier to read format can be found at https://choosealicense.com/licenses/mpl-2.0/
In essence: I chose this license because I want people that modify the code to give back to the community. The MPL-2.0 license makes sure that happens, protects me against patent trolls and still allows people to use the library as part of their own larger work & distribute that larger work with a license of their choice.
*) Disclaimer - I’m not a lawyer and this is not advice.