Wednesday 16 October 2019

Web APIs - A timeline of web services (Part IV)

Welcome to the world of lambdas and serverless functions

Microservices offer a highly-focused (single responsibility), low latency, auto-scaling web services. We could orchestrate these services as per our need. Developers look at microservices in not only a synchronous way but also in an asynchronous way.

One could tie an event-driven pattern using CQRS back-ending the microservices. A neatly designed CQRS can gracefully decouple read data to the write data. (That's for another post though)

Be it vert.x, sinatra or any other platform, we define services with the granularity we need.

Recently, for building a chat bot, I started exploring AWS lambda functions. Chat bot APIs need a very structured handling of states and transitions. So such APIs need to use session and context linking/sequencing to provide data interactively. I have been using nodejs serverless for building these APIs. Here's a quick guide to how serverless environment can be configured to make our services discoverable, secure and scalable.

I have to tell you that the world of APIs ever since I started my career always interested me. There is a choice of stack and a configuration that fits each of the clients needs in different ways.  An enterprise grade banking API needs are very different from that of a smart home automation IoT API.

Monday 29 July 2019

Web APIs - A timeline of web services (Part III)

RESTFul architecture re-imagines Web Service integration in a more http-oriented way. This kindled interest in the API designers & practitioners to fill in some gaps in standards-based metadata overload, security and client integration.

In RESTful Architecture, objects are modelled as resources (actors) as opposed to verbs/actions. It is typical in this mode of communication for the client to access and explore a resource before editing/modifying the data. For example, one GETs a user by searching with the id or name. Checks the address and modifies a field let's say street and PUTs. While the objects depend on the application/business/domain, the http actions are universal! Clients don't need to refer to getUser, editUser or deleteUser separately. Add attachment handling, header management, content type management, caching and metadata handling to this, it is a great recipe of APIs. No wonder most services are at peace (RESTful) these days.

The ease of REST services aligned naturally with the integration to web layers because of it's closeness to HTTP.

Anyone thinking swagger or apiary?

All Web Service / API Developers run into a common problem.
How granular should my service be? Do I have a mammoth service that does every possible activity at just 1 end-point? Or... do I do finely grained single responsibility super-responsive services that I can tie up as needed?

Continued in the next part...

(Please let me know your feedback in the comments.)

Tuesday 19 September 2017

Web APIs - A timeline of web services (Part II)

We have web services that relay data back and forth wrapped up with the metadata only just as much as needed.

That sounds fair enough!

What are these APIs?
Aren't these the documentation (manuals) for how to use any system? A system can be a database system or a bank account management system or an ERP system.

10 points to Griffindor for bravery and mainly compassion for saving us from the perils of defining the API!

Absolutely! The Shopping APIs or the Search APIs you see at Google or eBay are simply the Web Service APIs, shortly Web APIs.

Let's walk through the evolution of Web APIs.

The advent of XML eventually led to a nice protocol called SOAP. SOAP facilitated data access by wrapping data into understandable objects which are represented in an intuitive mail-like structure (imagine head, body and attachments) with a clearly defined metadata - schema.

Around 2000 and over a decade later, Web API world is taken by a Service-oriented Architecture (SOA) storm. This perspective of designing web/system architecture resulted in many advancements (Note: I'm using the term web with caution because its pervasive expansion to different devices resulted in broadening its own definition )
  1. separation of concerns through layering/composing services at different granularity, 
  2. security layering through OAuth and other mechanisms, 
  3. service discovery through a registry (UDDI)
  4. service definition and publishing standards (WSDL) in enterprise integration
to name a few.
Here's some more reference information on this.

Enterprise integration imbibed elements from SOA standards. WS-I OASIS and metro stack operationalized some standards defined and advocated by SOA working groups. The gamut of services and the complexity in defining an overarching integration framework resulted in exploring simpler components for integration. SOA (though it sustains) needed to uncomplicate.

A RESTful Architecture fills-in some of the short-comings of SOA.


Continued in the next part...

(Please let me know your feedback in the comments.)

Monday 18 September 2017

Web APIs - A timeline of web services (Part I)

Let me start from: How do we access data from a remote system?

In the world of connected systems, where there are all kinds of possible message exchange systems, talk about peer-to-peer, electromagnetic waves (radio, IR and all other types), bluetooth, wifi, there are numerous possibilities of how we can exchange messages with other systems.

As a message exchange standard across www, HTTP stands out as the most used. HTTP stood the test of time and scale! It has perpetuated decades of internet revolution while evolving to fit the needs of computing infrastructure and message exchange.


Web page:

Message exchange over a period of time became synonymous to web page request and response paradigm.

Here's a URL: https://google.com

And the server responds with a beautiful web page which satisfies all your urges for knowledge and wisdom!
This is how most internet services came into existence.
Seek (GET) what you need.
Wait... Are you registered? Sign up (POST).
Only then you see your data!

Session Handling:

Our needs went beyond the request-response (question-answer) style. By nature, we need more than an exchange of information - the context. This is when this miraculous state-saving/state-tracking/state-persisting technology came into existence.


Web Services and web APIs

While the world is happy with the life's answers to our needs through web pages, users asked the providers, "Do I need the overhead of your styles and showmanship when what I am actually using is the fields... only the fields I'm interested in?"

That's when the providers started thinking "Why do I need to have this guy's session on my page? I'm doing so much to keep him happy on the page with all my wooing experience! And still he/she is just happy with the fields".

A superb semantic medium called XML got the users and the providers on the same page

I need
<datafield1>value1<datafield1>
<datafield2>value2<datafield2>

And this is definitely more than what we need
<html>
<head>
<link...>
<link...>
<script...>
</head>
<div>datafield1</div>
<div>value1</div>
<div>datafield2</div>
<div>value2</div>
<body></body>
</html>

When the user doesn't need to stay logged in to their session, the server does not need to do session handling!

  1. More time for server. Less processing because of no session handling and asset manipulation. 
  2. Faster turnaround. Avoid context switching overhead. 
Question: How is it possible? 
Answer: A web service! 

You want to search products using a keyword.

http://www.mysampletesturl.com/search?keywords=smart,phone

And pat comes a result:

<products>
<product>
<id>123</id>
<name>iPhone</name>
<description>coolest of smart phones is here!</description>
</product>
<product>
<id>124</id>
<name>LG Nexus 6</name>
<description>Another coolest of smart phones is here!</description>
</product>
</products>

But this still happens over HTTP! What's the big deal? 
OK, we don't have to process all the additional display components (scripts, heads and bodies). I give it to you! 

What more? 

Continued in the next part...

(Please let me know your feedback in the comments.)

Thursday 28 May 2015

Read files from the root directory of your web application ( webapp/*) - Java

Was thinking about a good way to load internationalized error properties file to the web application, a simple resource bundle.
After a bit of thought, decided to make it simply this way:

Place it in the webapp/*

But how the hell do we read it from the java application which runs from my src/main/java?
Copy that at webapp/WEB_INF/i18n/*.properties and try reading the file at  ../../webapp/WEB-INF/i18n/<locale_name>/*.properties ?

Good luck!
Try that and let me know.

Life it not fair my friend and so are java programs on web apps!

Try this instead:
this.getClass().getClassLoader().getResourceAsStream("../i18n/<locale_name>/*.properties")

The classloader identifies the path starting from webapps\


Thursday 9 April 2015

Carnatic cassandra

Let's talk a bit about NoSQL with carnatic music. ;)

What if one has to represent the melakarta chart as a data model using an expandable schema and improve it for other use-cases? Here we go...

Melakartas define fundamental scales that are used in carnatic music (a classical musical form from southern part of India).

I'll start from defining a schema like this (An image below for our reference)

My primary key is going to be a combination of name of the chakra (the segments of the circle - indu, netra, agni and all) and the melakarta_id.

create table melakarta(chakra text,melakarta_id int,melakarta_name text,notation list<text>,shudhamadhyama boolean,primary key (chakra,melakarta_id));





Get some records in place. 

As a lighter side note, if you observe the names of melakartas like one of my colleagues did, carnatic system gives a lot of prominence to women.

insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',19,'Jhamkaradhwani',['S','R2','G2','M1','P','D1','N1','S'], true);
insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',20,'Natabhairavi',['S','R2','G2','M1','P','D1','N2','S'], true);
insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',21,'Keeravani',['S','R2','G2','M1','P','D1','N3','S'], true);
insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',22,'Karaharapriya',['S','R2','G2','M1','P','D2','N2','S'], true);
insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',23,'Gowrimanohari',['S','R2','G2','M1','P','D2','N3','S'], true);
insert into melakarta (chakra,melakarta_id,melakarta_name,notation,shudhamadhyama) values ('veda',24,'Varunapriya',['S','R2','G2','M1','P','D3','N3','S'], true);
In such a schema, each super row has 6 column families - as cassandra calls it - each with 6 melakartas.

How do we add ragas to this schema?

Multiple ragas can be derived from same melakarta. How can we store multiple ragas under each melakarta? 
Thanks to the collections (all your sets, lists and maps) in cassandra. We can always add another column in cassandra which has the set of ragas. I know you already noticed the list of notations. I'm just showing off!  

create table melakarta(chakra text,melakarta_id int,melakarta_name text,notation list<text>,shudhamadhyama boolean,ragas set<text>,
primary key (chakra,melakarta_id));

One thing to note about these column entries is that they insert keys in the sorted order automatically. Easy search, folks, easy!

If one were to design the same schema in an RDBMS world, it should probably look like this


create table melakarta(chakra text,melakarta_id int,melakarta_name text,notation list<text>,shudhamadhyama boolean,primary key (melakarta_id));

create table raga(raga_id int,raga_name text,melakarta_id int,primary key (raga_id));


If you were to pick which melakarta a raga belongs to, you would end up doing 2 lookups. Search table: raga and use the key melakarta_id to fetch the melakarta_name again. 

In a noSQL system, we highly encourage duplication.


create table raga(raga_id int,raga_name text,melakarta_name text,primary key (raga_id));
Make your queries less complex by bringing data to the same data model instead of incorporating FK's. This is a fundamental shift from the world of normalization. 

We thus knock some sense into the unstructured world with a little bit of compromise - duplication.

Sunday 15 March 2015

Sudoku Generator Initial thoughts (Incomplete)

Sudoku is a great distraction during a busy day. I somehow manage to stay away from this section at a cozy corner of the sports section on "the Hindu".

What if we were to design a sudoku generator which generates a fully formed sudoku template and simply plucks some numbers away depending on the difficulty level?

There are different ways in which we can generate a filled template. I'm thinking about one such (possibly) brute force way here. Comments are welcome!

1) Start from a random first line

4 7 5 | 3 2 8 | 6 9 1 ||


You can do this in 9! - 9 factorial - ways. (Sigh!)

2)

i)

Construct the next line in such a way that you pick the following 3 from the last 6 in the first row.

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | -  -  - | -  -  - ||  OR 2 6 3 | -  -  -  | - -  - ||

6P3 ways (6 permuted in 3 slots)

ii)
Repeat the same for the second and third slots

Note: When we are doing this, we have to look for repetitions and consistency.  

For example: 

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 5 1 | -  -  - || (This is wrong!)

Picked 4 5 1 from 4 7 5 and 6 9 1. This is clearly wrong.

It's also possible that we go about doing this over and over. While doing this we need to do some cool swaps while doing the validations in the same row.

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 5 9 | 7 2 6 ||  (This is wrong too!)

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||  (This is OK. Upphhh! Let's see...)


3) Pick the next 3 in third row in each 3 x 3 square is pretty much decided. It's only a matter of moving them around. We choose any order. 3 x 3!  

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 || 

With these 3 steps, we can generate the first 3 x 9 section. 

4) Try steps 1 - 3 vertically.

Pick the next 6 from 3 x 2 
7 5
8 1
6 9

Eg., 
4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5
7
9
1
8
6

5)  Pick the next 6 from 
4 5
3 1
2 9

now with validations, of course

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4
7 1
9 3
1 5
8 2
6 9

6)  We're not left with many choices for the next 6. The remaining 3 of each 3 x 2 square can be permuted. 

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2
7 1 8
9 3 6
1 5 4
8 2 7
6 9 3

7) It gets tricky from here on... 
In the second 3 x 3, vertically down, we need to pick 6 from
7 1 8
9 3 6

While placing the 6, we need to consider all the basic validations

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8
9 3 6
1 5 4
8 2 7
6 9 3

8) 
While trying to arrange 
9 3 6 
1 5 4 

We run into this strange issue where we need to find a not-so-straight-forward swap for 6. 

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 4 | 2 3 6 ||  (This is wrong!)
9 3 6
1 5 4
8 2 7
6 9 3

6 cannot be in the 1 x 3 section  2 3 6 because in the section
9 1 6
2 3 6
We need to have 1 - 9 each only once!  
What do we swap 6 with? 4 in 5 9 4? 
Let's see...

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 3 4 ||  (This is wrong!)
9 3 6
1 5 4
8 2 7
6 9 3

Oops! How can 4 be there? The vertical
1
5
4
6
4
Has two 4's now. Strict no-no!

Swapping with 3, 

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 4 3 ||
9 3 6
1 5 4
8 2 7
6 9 3

Good here!

9) 
Fill the next 3 in the 2 sections
7 8 3
5 7 9 
and
9 1 6
2 4 3

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 4 3 ||
9 3 6 | 2 1 4 | 5 7 8 ||
1 5 4
8 2 7
6 9 3

10) 

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 4 3 ||
9 3 6 | 2 1 4 | 5 7 8 ||
1 5 4 | 8 3 2 | 7 ----|| (This is a blunder again. 7 can't be here. So swap with 2? )
8 2 7
6 9 3

4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 4 3 ||
9 3 6 | 2 1 4 | 5 7 8 ||
1 5 4 | 8 7 2 | 3-----|| (Swapped with 3. 3 also has the same problem. So do 6 9 3)
8 2 7
6 9 3


4 7 5 | 3 2 8 | 6 9 1 ||
3 8 1 | 4 6 9 | 7 2 5 ||
2 6 9 | 1 5 7 | 3 8 4 ||
5 4 2 | 7 8 3 | 9 1 6 ||
7 1 8 | 5 9 6 | 2 4 3 ||
9 3 6 | 2 1 4 | 5 7 8 ||
1 5 4 | 8 7 2 | 3-----||
8 2 7
6 9 3

This is where we will have to do more smart swaps. Check-mate!
Time up. 
Will continue a little later.