What JSON?

If you’re one to consume JSON api’s (who isn’t really?), then it’s likely you’ve bumped into the issue of figuring out whether the resource you just called is returning a single JSON object or an array payload. As per the JSON API Spec,

The document’s “primary data” is a representation of the resource
or collection of resources targeted by a request.

Primary data MUST be either:

a single resource object, a single resource identifier object, or null, for
requests that target single resources

an array of resource objects, an array of resource identifier objects, or
an empty array ([]), for requests that target resource collections

To help out all the Gophers out there, I created a simple little helper package to figure it out for you.

Checkout the code here https://gitlab.com/kylehqcom/whatjson.

It’s pretty simple really, so instead of having a dependency to my repo, just copy paste into your project instead.

Usage

Just call WhatJSON() with a body of type []byte and an optional prefix. The code will check the very next char after the prefix has been matched to determine whether the body is a object or an array. Go check out the code already!

Short n sweet, enjoy =]

Lessons in Golang - Goroutines and Channels

Recently I made publicly available an in memory scheduler written in Golang, https://gitlab.com/kylehqcom/kevin.

Writing code for public consumption is always something of a scary exercise. Knowing full well the critiques, scrutiny and chin rubbing that will come from your peers and non peers alike. But being an optimist I appreciate any feedback. Good or bad it is always greatly received as it all just helps with the learning. My goal here is to share some of that learning, mistakes and “a ha” moments I have found in developing with Go.

Long may it continue!

Background

In my own time I have been writing a new SaaS. It’s currently private but details will emerge as and when the beta rolls out. The project itself is made up of 3 distinctive parts, a getter, a cruncher and a sender. Starting with the “getter”, it became evident pretty early on that I would need a way to schedule “gets”. But such a scheduler would come in super handy later on with my “sender” too. I first took a look at the existing open source packages.

From what I found, most were overkill for what I needed. Sure AirBnB’s chronos and even the “trimmed” down version kala would do the job, but have way too much overhead for me at this time.

At the other end of the scale, most took a direct linux cron job approach. gocron will certainly repeat tasks but didn’t have a nice way to manipulate running tasks or retrieve details from these tasks for introspection. It’s at this point that I took inspiration from the above and rolled up my sleeves. For reference, take a look at Kevin’s source.

Goroutines and Channels

Because my own requirements didn’t require a persistent store to track running jobs, implementing a schedule pool was as simple as creating a map key’d on a string.

type schedules map[string]*schedule

This allowed for the returning of a schedule based on a generic name, easy enough. When adding a Job to a Schedule, after validation, the details of the job are submitted in a non blocking *Goroutine. This ensured that any surrounding code can continue to execute and the job would happily run in the background. On submission, if an ID was not passed, a generated ID was returned to introspect the job in future. The job runner itself utilises Golang’s built in

time.NewTicker(time.Duration)

to repeat the actual function calls assigned to the job. So nothing terribly scary or out of the ordinary happening here. Well that was until I wanted to use an ID to stop a running job. Although at the time my “knowledge” of the problem was a wee bit lacking, I was pretty confident I could get the desired outcome.

The question was, how can I access the values of a function running in a background Goroutine?

As far as I was concerned “at the time”, each Goroutine is living in some “forbidden zone” away from variable scope. This left me with one option, using some well intentioned channels. Channels (among other things) are a way for Goroutines to communicate with each other. But how would I use a **stop channel for a pool of schedules with numerous jobs??? The internet is your friend.

After screening though numerous Stackoverflow results, my face made a rye smile when I finally found my way to @matreyer - I had been in contact with Mat earlier in the year in regards to a GoLang position so I wasn’t surprised to find his repo to handle my exact use case - https://github.com/matryer/runner Bam happy days! I now had a way to stop a Goroutine from the inside out.

But like most things, I wasn’t entirely happy with this initial implementation. I took issue with the duration between stop being called from the outside and the job timer checking for the stopped status. In short, a ticker is an infinite “for loop” in which you “break” out of or stop().

// do is called via Run in a go routine.
func do(r *runner) {
      // This will tick on "Every" forever.
      e := time.NewTicker(r.job.Every)
      for range e.C {
          ...
          // Work happens
          ...
          
          // But you can stop via a channel and 
          // break from the timer.
          if <- stop {
               break
          }

Take for example a job that gets an “Every” value of one week. If we are one day into the week and a stop is called on the job, the runner will then sit and wait for another 6 days before the ticker ticks and stops itself. I didn’t want that wait so set out to find a solution.

Curiosity for the win.

After hours of research, code attempts, Goroutine and channel examples I “happened” upon this playground example. Please go take a look.

https://play.golang.org/p/FZKVATjcu0

From this example, it is clear that Goroutines still have access to variables in package scope. Notice that var x int is being updated by one Goroutine, and then being rendered from another. Now that I know that Goroutines can access a variable in scope, then surely an out of scope variable can be replaced with a pointer instead?

A pointer is a memory address to a variable. In past code I had used pointers for memory optimisations or to ensure a single point of truth, I just hadn’t applied the same thoughts to Goroutines. So I’m sitting here thinking, if a Goroutine has access to a variable pointer from inside the ticker loop, then why do we even need these channels at all? Answer is you don’t!

*** My current implementation uses a pointer of a runner instance https://gitlab.com/kylehqcom/kevin/blob/master/runner.go#L172 and the runner instance is assigned against the schedule map via a JobID. This ensures that any updates to the stopped flag on the runner instance can be read by the ticker and will stop/break as required. Winning!

As much as I enjoyed implementing @matreyer’s package, it’s much more enjoyable losing an external dependency!

The Lesson

Channels and Goroutines are awesome and fun, but use them only when required. Often a simpler solution can be found. Even without Mat’s runner package, I am still bound to a ticker being stopped based on it’s Every duration. But I’m glad I looked for a better solution. I now have a deeper understanding and my Golang just levelled up.

*There are plenty of resources about Goroutines online so please “go” have a read if they are new to you.

** Interestingly, I could still have used a single channel sending JobID’s. Though as I currently see it, this would mean that every runner would get ALL job updates. JobID values would be sent down the channel and runners would have to self check themselves for a JobID match. Terribly inefficient.

*** Addendum: the current implementation uses the closing of an empty struct channel to exit the ticker.

29 Days in Murica No.2

Las Vegas

I certainly had expectations before entering Las Vegas. Most were met. But other experiences I encountered I hadn’t expected at all!

I mean the bar was well ‘n truly set exiting the plane “offramp”, walking through the airport doors and being faced with an entire wall of slot machines. Forget airport security, just give us some coin! The start of an underlying theme was beginning to occur.

Making our way from our gated “trailer park” community towards the “strip”, the shear scale began to bed in. Where else would you find an Eiffel tower outside of Paris, or a pyramid outside of Egypt?

In a photo frenzy we walk past the MGM, through Caesars palace, find the most ridiculously sized candy bars and rollercoaster our way between casinos. We continue past all the “minions” busking for photo moments, the stampede of “stag dooz” and “hen’s nights” that are all “so original”.

Finally reaching the strips’ end. With the constant fend against hustlers passing out their stripper flyers, we take refuge from all the flashy lights, and enjoy a moment to reminisce the Oceans’ Eleven closing scenes.

And then, after leaving the calm of the water cascades, in a subtle instant, it all becomes perfectly clear.

The whole thing just wreaks of desperation. And I found it to be really, really sad.

Barhumbug I know! Maybe if I was in my early twenties, drugged to the eyeballs with a fist full of dollars. On the lookout for shady shenanigans and the like, then this place makes a whole lotta sense. But that’s not what I saw here. Instead I saw middle aged men ++, clinging to some glimmering hope that their ticket or the next dice roll is “the” winner. That the extra forty pounds hanging around their midriff is sexually appealing. That the greasy slicked back hair is the way all the “kool kids” are doing it.

And regrettably, the desperation wasn’t gender specific. Although mostly paraded as eye candy in paying “professional” places of work, women acting as public patrons fell victim too. Plastic breasts that practically chin touch, aim for the sunrise or shoot for the stars. Short shorts and skirts that act as reminders of a cheerleading beer commercial career that “could have been”

Hmmmm…

Sometimes it’s just very hard to write about an experience without inferring a cast or judgement. I clearly have no prejudice against the people or actions I have seen. And I in no way have any knowledge of the hardships or experiences any of these people may have had. I guess I just got to look through the looking glass and came out confidently saying “that’s not the life for me”.

I am going to go an earn my successes, not gamble them. I only wish the same for you, whatever your successes may be.

29 Days in Murica No.1

Having recently spent 29 days in the good ol U S of A, I wanted to share just some of that experience with you. How better than to list 29 sights, sounds, emotions invoked and/or “things discovered” along the journey. *God knows I wanted and had every expectation to be writing these posts in transit. But to be perfectly honest, I’m actually glad I just let the road get in the way. To disconnect for a brief period of time and immerse myself in the vagrancy. So let us begin.

Pickles

HOLY SHIT does the USA know how to make a damn good pickle! That juicy bitter sharp sweet flavour that would always leave my eyes watering and an irresistible urge to just eat some more. Back in NZ we generally get wee little gherkins. As a nation we must end this hardship. We have too long been deprived and no longer can we endure this suffering!

GIVE US THE PICKLES!

But of course ‘Murica being what it is, they had to go next level on the pickle thing….welcome to FRICKLES! A.K.A Deep Fried Pickles! Just take look at this face if you want to know how good they are!

*Figurative speech only.

Someone got spoilt this year

Each year, we all get to enjoy travelling around the circumference of the sun a.k.a your birthday. Some take this time to reflect. Others try find a new youth and binge themselves at the bottom of a bottle or a handful of pills. This year, my lovely lady decided to surprise me and change it up!

Knowing I’d nudged a fair amount of speed via the 2 wheeled variety, and that I’d be petrified of jumping from a perfectly good plane, she did the next best thing. It comes very much recommended!!!

It's where the magic happens © 2018 - KyleHQ