Current Status

Actively playing with all possible backend services

22 August, 2018

Deploying Spring Boot application to Google Cloud App Engine

If you have worked with AWS and GCP, you should have understood that AWS' Elastic Beanstalk and Google's App Engine(GAE) enable deploying our applications without the need to understand how to build or scale the underlying infrastructure.

Their promise is great, and for most use cases as there are no servers to maintain. We simply upload our applications and they are ready to go.
If everything works as promised Beanstalk or the Google App engine will take care of scaling, load balancing, traffic splitting, logging, versioning, roll out and roll backs, and security scanning etc. Pretty simple , ah?

Today I am going to share simple steps to deploy your first Spring boot (with Java and Maven)  application to  Google Cloud App Engine.

So let's create s simple create a Spring Boot Java API on Google App Engine.

I asume you have set up Google Cloud SDK in your local. (https://cloud.google.com/sdk/gcloud). Alternatively you can use Cloud Shell.

$ gcloud auth list
$ gcloud config list

Bootstrap your application using SPRING INITIALIZR:
https://start.spring.io/


Unzip the file and open with your IDE:


Add the "Cloud SDK-based" maven plugin to the POM file:

<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>appengine-maven-plugin</artifactId>
<version>1.3.2</version>
</plugin>




Make sure to remove spring-boot-starter-tomcat dependency if it is there in the POM.
Spring Boot also includes a JUL to SLF4J bridge might interferes with App Engine's log handler. To make sure we get proper log entries, let us exclude the jul-to-slf4j artifact by marking it's scope as "provided".

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
    <scope>provided</scope>
</dependency>


Also add javax.servlet-api dependency as provided

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>


Then adding App Engine Descriptor is the most important task of this:
App Engine Java applications use Descriptor file (another configuration file), named appengine-web.xml, to specify the app's registered application ID(unused for Cloud SDK based tooling) and the version identifier of the latest code:

we must add create a new src/main/webapp/WEB-INF/appengine-web.xml descriptor file:

$ mkdir -p src/main/webapp/WEB-INF/
$ touch src/main/webapp/WEB-INF/appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>_our_app_id_</application><!-- unused for Cloud SDK based tooling -->
  <version>1</version><!-- unused for Cloud SDK based tooling -->
  <threadsafe>true</threadsafe>
</appengine-web-app>




get the "OUR_PROJECT_ID" from the command line:

$ gcloud config list --format 'value(core.project)'

Ok, let's get to your Spring project and write a simple API:




you can find sample code here : https://github.com/belpamulle/springboot-appengine

Let us run the app in local:
mvn package
mvn appengine:run

if you get this error:

Failed to execute goal com.google.cloud.tools:appengine-maven-plugin:1.3.2:run (default-cli) on project bankapi: Dev App Server does not support App Engine Flexible Environment applications. -> [Help 1]

make sure project's Packaging = war



Deploying the Application into App Engine:

1) Initialize the project to run in the your desired region:
$ gcloud app create --region australia-southeast1

2)Then, deploy your application into App Engine environment, run :
$ mvn appengine:deploy

Now we can visit the API at http://{OUR_PROJECT_ID}.appspot.com in your web browser.



Success ! :)

Clean up:
You can Disable your application/Delete your project



***
Note that behind the scene App engine uses Jetty web server underneath and runs on Google Kubernetes Engine, so you should check Google Kubernetes Engine pricing if you are going to leave your app running after testing or make sure to clean up your work. The environment should be scale down to 0 instances when no one is using it, and automatically scale up when traffic is coming.

21 August, 2018

com.google.appengine vs com.google.cloud.tools appengine-maven-plugin

I was working on a Spring boot application which I needed to deploy in Google Cloud Appengine, in the same time one of my colleague made a pull request for a similar code. Then I noticed that we two have used 2 different Maven plugins from Google appengine with same artifactId.

they are:

<plugin>
<groupId> com.google.appengine</groupId>
<artifactId> appengine-maven-plugin</artifactId>
<version> 1.9.64</version>
</plugin>


and

<plugin>
<groupId> com.google.cloud.tools</groupId>
<artifactId> appengine-maven-plugin</artifactId>
<version> 1.3.2</version>
</plugin>

at the time of writing followings are the 2 articles I could find in Googledocs for 2 plugins

 https://cloud.google.com/appengine/docs/standard/java/tools/maven
 https://cloud.google.com/appengine/docs/standard/java/tools/using-maven

 So as the name suggests they are "App Engine SDK-based" and "Cloud SDK-based".

After doing some experiments I feel that in most cases the second one was convenient to use. 2 approaches have different authentication and in second option I could the same credentials of my Cloud SDK-based commands, which I used for gcloud auth login flow.

 Please note that second plugin need us to have Google Cloud SDK installed in your local.

10 August, 2018

My Docker learning short notes

I had this note from long time in my Evernotes, I thought I should have post it here, so someone might find it useful.

Why docker?

Containerization! Running applications in containers instead of virtual machines.
Dockers are not VMs. Different benefits. One of the main advantages to a Docker-based architecture is standardisation. Also it ensures Continuous deployment  from development to production with CI efficiency to (build a container image and) use  same image across every step of the deployment process.

Pros:
Self contained
Isolated environment
Runs almost everywhere(specially in the Cloud)
Small and lightweighted
Very scalable

Commands:

Get the version
$ docker --version
or
$ docker -v
$ docker-compose --version
$ docker-machine --version

$ docker run hello-world
$ docker run -d -p 80:80 --name webserver nginx
http://localhost/
$ docker container ls
$ docker container stop webserver
$ docker container ls -a
$ docker container rm webserver
$ docker image ls
$ docker image rm nginx

What is an image?

An image is an executable package that includes everything needed to run an application--the code, a runtime, libraries, environment variables, and configuration files.

What is an Container?

A container is a runtime instance of an image--what the image becomes in memory when executed (that is, an image with state, or a user process).

Dockerfile?

Portable images are defined by something called a Dockerfile

We can create a file called Dockerfile (contents differ).
To create a Docker image run the build command with tag using -t so it has a friendly name.

$ docker build -t friendlyhello .

To run the app, mapping your machine’s port 4000 to the container’s published port 80 using -p:

$ docker run -p 4000:80 friendlyhello
docker build -t friendlyhello .  # Create image using this directory's Dockerfile
docker run -p 4000:80 friendlyhello  # Run "friendlyname" mapping port 4000 to 80
docker run -d -p 4000:80 friendlyhello         # Same thing, but in detached mode
docker container ls                                # List all running containers
docker container ls -a             # List all containers, even those not running
docker container stop            # Gracefully stop the specified container
docker container kill          # Force shutdown of the specified container
docker container rm         # Remove specified container from this machine
docker container rm $(docker container ls -a -q)         # Remove all containers
docker image ls -a                             # List all images on this machine
docker image rm             # Remove specified image from this machine
docker image rm $(docker image ls -a -q)   # Remove all images from this machine
docker login             # Log in this CLI session using your Docker credentials
docker tag username/repository:tag  # Tag for upload to registry
docker push username/repository:tag            # Upload tagged image to registry
docker run username/repository:tag                   # Run image from a registry
docker logs containerName #check logs related to one container
docker stack ls                                            # List stacks or apps
docker stack deploy -c   # Run the specified Compose file
docker service ls                 # List running services associated with an app
docker service ps                   # List tasks associated with an app
docker inspect                    # Inspect task or container
docker container ls -q                                      # List container IDs
docker stack rm                              # Tear down an application
docker swarm leave --force      # Take down a single node swarm from the manager

Multi-container, multi-machine applications are made possible by joining multiple machines into a “Dockerized” cluster called a swarm. A swarm is a group of machines that are running Docker and joined into a cluster.


go inside container:
docker exec -it {id or name} sh
docker exec -it {id} bash

Getting IP address of docker container running:
docker container ls
docker inspect | grep "IPAddress"

24 July, 2018

Learnings - Kong API Gateway

 Hi folks, Many days I was playing with Kong in Docker, quick scaffolding for anyone interest to play with it. This summarises my basic learnings.
What is Kong?
Kong is an API gateway. Some of the popular features deployed through Kong include authentication, security, traffic control, serverless, analytics & monitoring, request/response transformations and logging.
A typical Kong setup is made of two main components:
1)Kong’s server, based on the widely adopted NGINX HTTP server
2)Kong’s datastore, Apache Cassandra and PostgreSQL can be used.

Terminology:
What are Kong plugins?
Kong API gateway features are provided by plugins. Authentication, rate-limiting, transformation, logging etc, are all implemented independently as plugins. Plugins can be installed and configured via the Admin API running alongside Kong.
What is a Kong Service?
a Service is the name Kong uses to refer to the upstream APIs and microservices it manages.

What is a Route?
Routes specify how (and if) requests are sent to their Services after they reach Kong. A single Service can have many Routes.

Who are Consumers?
Consumers are associated to individuals using Kong APIs, and can be used for tracking, access management, and more.
What is an Upstream service?
Upstream service refers to API/service sitting behind Kong(our backend service), to which client requests are forwarded.
What is an API?
API is a legacy entity/term used to represent your upstream services. Deprecated in favor of Services since 0.13.0.

Running it locally: 

Steps

  1. Create a Docker network:
>docker network create kong-network
(you can have any name for "kong-network") 

2. Run Kong DB :
run Postgre DB ( here I have used Postgre, but you can use Casandra also)

>docker run -d --name kong-database \
--network=kong-network \
-p 5432:5432 \
-e "POSTGRES_USER=kong" \
-e "POSTGRES_DB=kong" \
postgres:9.6
*Docker tip: next time (start): docker start kong-database or you can start with process id
Eg:
>docker ps -a
>docker start 8c960a6b930f

3. Configure database /run the migration
>docker run --rm \
--network=kong-network \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
kong:latest kong migrations up

*using --rm switch - container gets destroyed automatically as soon as it is stopped

4. Run Kong on docker in same network

>docker run -d --name kong \
--network=kong-network \
--expose=8000 \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
-e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
-p 8000:8000 \
-p 8443:8443 \
-p 8001:8001 \
-p 8444:8444 \
kong:latest
test:
>curl -X GET http://127.0.0.1:8001/
5. Creating dummy endpoint for testing

You can use any HTTP rest endpoint mocking tool, I prefer https://www.mocky.io/
***Note: if you want to test your own API running in separate docker container, make sure that API docker container is in the same network (which makes 'localhost' access to Kong container)
or simply add 'Service' to Kong with the IP as the 'host'

6. Add a service
>curl -i -X POST http://localhost:8001/services/ \
-d "name=mockyService1" \
-d "protocol=http" \
-d "host=mocky.io" \
-d "port=80"
or all-in-one :
>curl -i -X POST http://localhost:8001/services/ \
-d "name=mockyService1" \
-d "url=http://www.mocky.io:80"
test:
>curl -X GET http://127.0.0.1:8001/services/

delete a service?
>curl -X DELETE http://127.0.0.1:8001/services/ac0582e9-40d5-49bc-b711-615216d51b5f

7. Add routes
>curl -i -X POST http://localhost:8001/services/mockyService1/routes/ \
-d "protocols[]=http" \
-d "hosts[]=requesthostname.com" \
-d "paths[]=/v2/5b5558d3320000b804827d47" \
-d "strip_path=false"
instead, you could use the JSON format:
>curl -s -X POST http://localhost:8001/services/mockyService1/routes/ \
-H 'Content-Type: application/json' \
-d '{ "protocols": ["http"], "hosts": ["requesthostname.com"], "paths": ["/v2/5b5558d3320000b804827d47"], "strip_path": false }'
test:
>curl -i http://localhost:8000/v2/5b5558d3320000b804827d47 -H 'Host: requesthostname.com'

how to delete a route?
get id from :
>curl -X GET http://127.0.0.1:8001/services/serviceName/routes

>curl -X DELETE http://127.0.0.1:8001/routes/5c293b78-38d3-4129-b8b1-e7001e1d37c4

References:
Kong Docker installation:
https://docs.konghq.com/install/docker/

Kong Kubernetes installation:
https://docs.konghq.com/install/kubernetes/

Other installations:
https://konghq.com/install/
Kubernetes Ingress Controller for Kong :
https://www.youtube.com/watch?v=2LInu_ZiL3E

12 February, 2018

Mockito to execute or not a method


This is well known thing if you are using Mockito to mock your services. But I wanted to note it here.

 Imagine you have a method call inside method you are writing the unit test (or eg:-getPayment() ) that should not be called when your test is executed.
 That may be the first reason why you mocked it in the first place.

 In below code, I want to mock getPayment() method without executing it.

option A:
    Mockito.when(subject.getPayment()).thenReturn(paymentDetail);

option B:
    Mockito.doReturn(paymentDetail).when(subject).getPayment();

If you tested both, option A will actually call the getPayment()- method while the second will not. Both will cause doSomeStuff() to return the desired `paymentDetail` Object.

Hope this helps someone new ith Mockito!


Some Popular Posts