sábado, 28 de maio de 2016

Versioning APIs

I'm currently working in a project where my team is developing an API, and one subject that came to the table is versioning. This is a very confusing subject and generates a lot of discussion, that's why I had the idea to write a post about it.

There are a few strategies for versioning a API, but let us get a step before, what happens in the project that requires a new version of the API?

Contract break

Let's say we have a resource called Person, its contract includes id, name, birthDate, address, zipCode and city. At some point a decision is made to change it and separate the address information from the Person. It means that the contract will be changed because the address information will be moved to a separated resource.

All consumers will be broken, because they look at the address information inside the Person. This is a contract break. There are other cases where new information is added to the contract, which means it will not break any consumers, so we cannot consider this case as a contract break.

Version it!

How to avoid breaking the consumers? Versioning it. At this point, API team will create a version 2 of the contract. Consumers will still use the old contract, however a version 2 of the contract will be published. Consumers and API team will now have an agreement on when the API will stop supporting the old version and consumers will have to start using the new version.

Be cautious

Versioning contracts looks like a good solution, but it can get dangerous if the API starts supporting a lot of versions. It will make your code look like a mess, hard to understand, too many branches on the code. I won't even mention that it can cause bugs (just did it :P). I'd say a good practice would be accumulating your contract breaks and release a new API version once you evaluate it is worth to. Also, I wouldn't have more than two vesions in parallel to avoid the issues mentioned above.

Versioning strategies

I've been researching a couple of solutions for API versioning, I will present and comment two strategies that most called my attention.

Version as path/query parameter

This is the strategy I've most seen on projects I worked on and on my researches. It consists in adding the version in the path like the example below:


Who is using this approach?
  • Twitter
  • Atlassian
  • Google Search

Version as a header

This is probably the less intrusive strategy, where the version is informed in the header Accept, leaving the URL clear. See the example below:

Accept: application/json; version=1.0

There are other ways to inform the version in the Accept header, but I thought this one is the clearer way. I also saw some example where people use a custom header like X-Version: 1.0

Who is using this approach?
  • Azure
  • Github API
  • Google Data API


A good contract design many times avoids contract breaks, which avoids versioning. Always be careful about the contracts, these set how the external world talk to your API. Contracts break, it's natural, however, always evaluate each change, think about your design and ask yourself if each change is really the right thing to do, collect the pros and cons and do smart decisions.

quinta-feira, 26 de maio de 2016

Integrating node.js and Apache Kafka

In this post I will demonstrate how we can integrate node.js and Apache Kafka, producing and consuming messages in a very simple example.

First of all, let us get Apache Kafka up and running, you can see how to do on the official kafka's site tutorial.

Once it's up and running, we can set up the project and start playing with the lib no-kafka

  • npm init
  • npm install no-kafka --save

I have used the version 2.4.2 of no-kafka. So, if you want to inform the version when installing, just run it as "npm install no-kafka@2.4.2 --save".

Here is a producer example, which will connect to kafka and produce messages in a topic.

var Kafka = require('no-kafka');
var producer = new Kafka.Producer();
return producer.init()
  return producer.send({
      topic: 'kafka-test-topic',
      partition: 0,
      message: {
          value: 'Hello!'
.then(function (result) {
  console.log('topic sent');
If you are running a local instance, it connects automatically to the local host. To connect to a external instance, you can replace "localhost" by the external host following the example below:

var Kafka = require('no-kafka');
var connString = ' kafka://localhost:9092, localhost:9092 '
var producer = new Kafka.Producer({ connectionString: connString });

Here is a consumer example, which will connect to kafka and subscribe to a topic, receiving messages and printing them to the console.

var Kafka = require('no-kafka');
var consumer = new Kafka.SimpleConsumer();
// data handler function can return a Promise 
var dataHandler = function (messageSet, topic, partition) {
    messageSet.forEach(function (m) {
        console.log('topic received: ');
            'partition': partition,
            'offset': m.offset,
            'message': m.message.value.toString('utf8')
return consumer.init()
.then(function () {
    return consumer.subscribe('kafka-test-topic', 0, dataHandler);

As you can see in the pieces of code above, all requests return promises. This is an example of the very basic features of the lib interacting with Kafka.
I put this project on my github, so then you can play with the code and evolve as needed.

Building APIs with HarvesterJS

HarvesterJS helps creating robust APIs on the top of mongoDB and node.js. It is a fork of fortuneJS and is JSONAPI compliant, and runs under Express. It gives the developer the ability to create contracts and validations with Joi.

In this post you'll see how to setup a very basic API with schema validations and some features of HarvesterJS.

Once the resources are properly set up, HarvesterJS provides the GET/POST/PUT/DELETE operations persisting the data on MongoDB.

Initial project setup

  • npm init
  • npm install harvesterjs --save
  • npm install joi --save

Seeting up the API with configs


var harvester = require('harvesterjs'),
    options = {
        adapter: 'mongodb',
        connectionString: 'mongodb://',
        inflect: true
var harvesterApp = harvester(options);


function onListen() {
    console.log('listening on port 4567');

harvesterApp.listen(4567, onListen);

Setting up a resource

You can setup the resource fields and use JOI to describe and include validations on the field. In the examples below, we have a resource called customer with two fields: status and name

  • Status is a string which only accepts two values: Active or Inactive
  • Name is a string which is required.

Basic resource customer.js

var Types = require('joi');

harvesterApp.resource('customer', {
    status: Types.string().valid('Active', 'Inactive'),
    name: Types.string().required()

Linking resources

Regular link

In the example below, we have a resource called customer which has a link to a resource called contact. This link is an array of contacts, but you can have a single resource link.

var Types = require('joi');

harvesterApp.resource('customer', {
    status: Types.string().valid('Active', 'Inactive'),
    name: Types.string().required(),
    links: {
       contacts: ['contact']

External link

In the example below, we have a resource called customer which has an external link to a resource called contact.

var Types = require('joi'),
   contactURI = 'http://localhost:2426/contacts';

harvesterApp.resource('customer', {
   status: Types.string().valid('Active', 'Inactive'),
   name: Types.string().required(),
   links: {
      contact: { ref: 'contact', baseUri: contactURI }

Manipulating resources manually

HarvesterJS gives you the ability to manipulate documents manually. Once you have the harvesterApp object in place, you can use harvesterApp.adapter's methods to interact with mongoDB: find, findMany, create, update, delete.

These are the very basic features of HarvesterJS, for more information check its github.

domingo, 22 de maio de 2016

Funcional Programming with Python

I've been playing with functional programming for some years. I learned languages like Haskell and Scala which help us doing really nice functional things.

You can notice that I mentioned two languages which we can do functional programming, but not both of them are purely functional languages. Haskell is a pure functional language, however Scala is a hybrid language where you can do both of functional and/or object oriented programming. Some people don't like these hybrid languages just for the fact that you can mix both paradigms and actually do wrong things.

In my humble opinion, I don't see big problems, since you know what you are doing and your team have good practices like code review and design review.

This is a big discussion, I'm giving my opinion and I will try to prove that we can do really nice functional programming even with a hybrid language, like Python.

A bit of functional programming

Below are some characteristics of functional programming:
  • Expressions over statements (instead of using a whole for statement, use map/filter/reduce)
  • No side effect
  • Immutability
  • Simpler code
  • Expressiveness
  • Composable code

Functional is Stateless


a = 0
def plus_one():
 a += 1

def plus_one(a):
 return a + 1

Don't iterate over lists

numbers = [1,2,3,4,5,6,7,8,9]
odd = []
for n in numbers:
 if n%2:

Use map/filter/reduce

def isEven(n):
 return n%2 == 0

numbers = [1,2,3,4,5,6,7,8,9]
odd = filter(is_even, numbers)

An even better way

numbers = [1,2,3,4,5,6,7,8,9]
odd = filter(lambda n: n%2 == 0, numbers)

An example with map

numbers = [1,2,3,4,5,6,7,8,9]
squares = map(lambda x: x * x, numbers)

An example with reduce

numbers = [1,2,3,4,5,6,7,8,9]
numbers_sum = reduce(lambda x, y: x + y, numbers)


greet = lambda name: 'hello {0}'.format(name)
print greet('cure')

High Order Functions

  • Functions that take function(s) as argument
  • Functions that return a function

def compose_func(func1, func2):
 return lambda x: func1(func2(x))

def build_engine(power):

def build_body(engine):

build_car = compose_func(build_body, build_engine)

List Comprehentions

  • Comes from mathematics
  • { x2 | x ∈ ℕ }
  • x * 2 given x is contained on the set of natural numbers

a = [x**2 for x in range(10)]
b = [2**i for i in range(13)]
c = [x for x in a if x%2 == 0]


You don't need a pure functional language to do functional programming, you can follow its principles by thinking functional when coding. That's what I tried to prove with the examples above. I hope it helps you.