Creating a docker instance on AWS using Ansible.

There are many advantages of infrastructure automation. We can implement any software project without worrying about the system configuration.

This project utilizes ansible for creating an ec2 instance dynamically and then creates a docker instance on the remote ec2.

You can simply clone the repository in your local machine and create an ec2 instance just by changing the parameters.

Prerequisites

python3
ansible 2.5
ubuntu 18.04 LTS

STEP 1: Add AWS login details in the  ‘vault.yml’ .

AWS login credentials are required to create the ec2 instance. These can be stored in the ~/.aws directory or this can be added into the ansible vault using ‘ansible-vault’ command.

Execute below command and add aws_access_key_id and aws_secret_access_key in the file created by the command.

$ sudo ansible-vault create vars/vault.yml
[default]
aws_access_key = XXXXXXXXXXX
aws_secret_key = XXXXXXXXXXXXXX

STEP 2: Initialize AWS variables for the configuration of ec2 instance.

vars/vars.yml contains variables like AWS- region id, VPC id, subnet id which is essential for creating ec2. These should be changed according to the AWS login.

aws_region_id : eu-central-1
aws_vpc_id : vpc-07db38c12d26359ac
aws_sub_id : subnet-042581e3da666d445
aws_ami_id : ami-050a22b7e0cf85dd0

aws_instance_type : t2.micro

aws_sec_key : abhishek-vpc

IdFile : /home/abhishek/.aws/abhishek-vpc.pem

default_image : jenkins

public_ip : ''
instance_id : ''

terminate : true

public_ip and instance_id are variables that is initialized at the runtime.
The variable ‘terminate’ can be set to true or false, depending on the final state of the instance.

The variable default_image can be changed based on the required image.
This image is pulled from https://hub.docker.com/

STEP 3: Execute the ansible-playbook to create docker on ec2.

$ sudo ansible-playbook create_ec2.yml -i hosts.ini --ask-vault-pass

The command will ask the vault password which is created in the STEP1. It will create a docker on ec2 based on the configuration available in create_ce2.yml.

Acknowledgments

• Erika Heidi – Digital Ocean – anisble-cheat-sheet
• C Tarwater – chrisanthropic.com – ansible-ec2-aws

 

Creating Exasol Docker Instance

Introduction

Exasol is an analytics database management software, this is an in-memory, column-oriented, relational database management system.

It supports  SQL Standard 2003 and can be integrated via standard interfaces like ODBC, JDBC or ADO.NET.

Creating Exasol Docker instance

Exasol docker instance can be created with the help of GitHub page.  This requires docker tool installed on the system.

host-machine:~$ sudo docker run –name exasoldb -p 8899:8888 –detach –privileged –stop-timeout 120 -v exa_volume:/exa exasol/docker-db

The command above will create exasoldb container. It will expose exasol container port 8888 on the host machine at port 8899. It means exasol database would be available at connection string host-system:8899.

It will also configure exasol database in persistent mode. i.e., the objects created will remain stored in the system,

Starting the exasol instance

The command in the last paragraph will create a exasol instance in a docker container. we can retrieve the container id using below command.

host-machine:~$ sudo docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d4f553b20771 exasol/docker-db “/usr/opt/EXASuite-6…” 6 days ago Up 53 minutes 0.0.0.0:8080->80/tcp, 0.0.0.0:8899->8888/tcp exasoldb

d4f553b20771 is the container id. This is a reference to the container which is running the exasol. Use this id to start the container as given below.

host-machine:~$ sudo docker container start d4f553b20771 

Stopping the exasol database gracefully 

Exasol database is a system which needs to be stopped carefully. If the host-system is shutdown with stopping the database. The tables created may be lost or system may not start again.

host-machine:~$ docker exec -ti exasoldb dwad_client stop-wait DB1

The command above will stop the database container. The host-machine can be shutdown or reboot.

Connecting using sql client.

A JDBC driver can be downloaded from exasol official download.  I used Dbeaver to connect. Default user name is : sys and password is exasol.

Note – Docker image is not officially supported by exasol, but still can be used for simple use-cases.

 

 

 

CMUS – A command line music player.

cmus is command line based music player for linux based operation system. It supports various output methods by output-plugins. It can be controlled from outside using cmus-remote.

We can use cmus for playing audio on local drive, audio hosted on network or any live stream. I have used cmus to play online radio stations.

There are 7 views in cmus. Press keys 1-7 to change active view.

Library view (1) , Sorted library view (2) , Playlist view (3) , Play Queue view (4) , Browser (5) ,Filters view (6) , Settinsgs view (7)                                                                                                                               Multiple Views of cmus

Cmus Control                                                                                                                                       cmus is conrolled using commands and key presses. Press “:” to enter comand line and “Esc” to come out of command mode. Commands can be auto completed by pressing “tab”.

Press “a” in command mode to add any music to library. Use following key presses to copy marked or selected tracks from views 1-5

a copy tracks to the library in view (1-2)                                                                                          y copy tracks to the playlist in view(3)                                                                                                e append tracks to the play queue in view (4)                                                                                  E prepend tracks to the play queue in view (4)

Common cmus commands

q quit -i
:q exit cmus.
b player-next
c player-pause
x player-play
z player-prev
v player-stop
+ volume +10%
– volume -10%

cmus-remote

This is a utility to control running cmus from outside cmus process.  If no argument is given it connect to local cmus socket (~/.cmus/socket). It can also connect o external cmus using  “–server SOCKET”  argument.

Some common commands to control are

-p Start playing.
-u Toggle pause.
-s Stop playing.
-n Skip forward in playlist.
-r Skip backward in playlist.
-R Toggle repeat.
-S Toggle shuffle.
-v VOL Change volume.
-k SEEK Seek.
-Q, Get player status information
-l Modify library instead of playlist.
-P Modify playlist (default).

 

cmus playlist

You can create a playlist and put in cmus home. It will be visible in playlist view(3). Some of the bollywood channels are listed below.

pi@raspberrypi:~/.cmus $ cat > playlist.pl

http://50.7.77.114:8296/

http://174.36.206.197:8198/

http://198.100.149.27:8012/

http://uk2.internet-radio.com:8043/live

http://69.64.61.173:8057/stream

http://50.7.77.114:8296

http://5.9.66.201:8896

http://185.66.249.48:9600/stream

http://69.162.111.146:9960

http://192.240.102.133:8512/stream

 

 

MQTT Python Client

In this article, I want to create a python module which would connect to mosquitto server. We can connect a MQTT client as subscriber or publisher of message.

Mosquitto python module has been donated to Eclipse Paho project, which is open-source client implementations of MQTT.  This module can be installed using pip command.

$ sudo pip install paho-mqtt

Subs.py – The code given below is subscribe module. You can look at comment to understand in detail.

#The line imports python mqtt module.
import paho.mqtt.client as mqtt

#Here we initilize variables used in the code.
server = "192.168.0.XXX"
port = 8883
keep_live = 45
topic = "msgTopic"

# Define routines to register.
# Client - The instance at the client side.
# userdata - Data to be passed by the user.
# flags - response flag sent by the server.
# rc - Return code, 0 is successful connection.
# mid - message id
# granted_qos - Quality of service granted by the server, default 0.
# payload - the message received on the topic.
def on_connect(client, userdata, flags, rc):
 mqttc.subscribe(topic, 0)
def on_subscribe(client, userdata, mid, granted_qos):
 print("Subscribed to: "+str(mid)+" "+str(granted_qos))
def on_message(client, userdata, msg):
 print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))

# Initilize a client instance and ser password for the connection.
mqttc = mqtt.Client()
mqttc.username_pw_set(username="username",password="password")

# register the routines.
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe

# connect to the server and wait for message.
mqttc.connect(server, port, keep_live)
mqttc.loop_forever()

Pubs.py – The code given below is publisher module. You can look at comment to understand in detail.

#The line imports python mqtt module.
import paho.mqtt.client as mqtt

#Here we initilize variables used in the code.
server = "192.168.0.191"
port = 8883
keep_live = 45
topic = "msgTopic"

# Connect and publish modules
def on_connect(client, userdata, flags, rc):
 mqttc.subscribe(topic, 0)
def on_publish(client, userdata, mid):
 print "Message Published..."

# Initilize a client instance and ser password for the connection.
mqttc = mqtt.Client()
mqttc.username_pw_set(username="mqtt",password="mqtt")

# register the routines.
mqttc.on_publish = on_publish
mqttc.on_connect = on_connect

# Connect and publish the message on a topic.
mqttc.connect(server, port, keep_live)
mqttc.publish(topic,"Hello! How are you?")
mqttc.disconnect()

 Output – Below is the Publish and Subscribe module in action on raspberry pi.

MQTT

MQTT is simple publish subscribe message broker, which is used for IOT based machine to machine (M2M) communication. It is simple to use. It needs a MQTT server, which can be hosted on any linux or windows machine.

Execute The following line with root privilege, it would import security code and add to local key repository and install mosqitto service.

$wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
$sudo apt-key add mosquitto-repo.gpg.key
$cd /etc/apt/sources.list.d/
$sudo wget http://repo.mosquitto.org/debian/mosquitto-wheezy.list
$sudo apt-get install mosquitto

Making mosquitto secure – Add below lines to the file /etc/mosquitto/mosquitto.conf. This would make the mosquitto listen on port(8883) at all the interfaces connected to Raspberry pi.

listener 8883 0.0.0.0
password_file /etc/mosquitto/pwfile

Update the password file with the username you want to connect to the mosquitto service and restart the mosquitto service.

$ sudo mosquitto_passwd -c /etc/mosquitto/pwfile mqtt
Password:
Reenter password:
$ sudo service mosquitto restart
$ ps -aef | grep mosq
mosquit+ 2150 1 0 09:55 ? 00:00:00 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

 

Testing the service – You can use many clients, e.g. python paho-mqtt, android based client, online client. I have used chrome based app called MQTTLens. It can be found on chrome app store.

Add a New connection

Subscribe & Publish

To remove mosquitto run apt-get purge.

$sudo apt-get purge mosquitto

There can be multiple use of MQTT, I will use this to communicate with raspberry-pi, ESP8266 and Arduino modules in coming posts.

 

Raspberry Pi

This little credit card size ARM based computer has intrigued me since the time I held it first time in my hand. It just happened by chance, once while browsing through Amazon I ordered it and since then it has become my favorite.

This little piece of hardware gives wings to your imagination, you can do everything you wished right from OS level software experiments to hardware level things.

Raspberry Pi – 2 Model B

It has got 4 USB ports with RJ45 (LAN interface), HDMI and audio visual port. It needs a micro-sd card and external 5v supply. You can use any 5v, 1 AMP or above power supply. Any mobile charger with micro usb port would work fine. There are choice of OS available on here. However, I have used raspbian since beginning, because of support and help available online. Raspbian is debian based Linux distribution for raspberry pi.