Publish Docker image with Automated build

docker1

Docker is an open source tool that helps us to develop, ship and run applications.  Each and every application needs a Dockerfile. Instructions are written over Dockerfile in a layered/step-by-step structure to run the application. The Dockerfile is then built to create Docker image. We can then push the image either to Docker Hub or keep it in our Workstation. Finally to run the application we have to run container from the image which can be treated as a concept like an instance of the image since we can have more than one container of a single image. If the Docker image is not in our local Workstation we first have to pull the image from Docker hub and then run container from it. But when we need to make changes in Dockerfile(as needed to make changes in application) we have to build a new Docker image again from it. Or we can make change in the container we are running and commit it but that doesn’t reflect the Dockerfile which contains all the instructions to run an application say it for development or production purpose. Also building Docker image locally from Dockerfile makes us doing things like build, push and all manually. Automating build of Docker image from Dockerfile that is either on Github or Bitbucket is the only solution for this 🙂 .

In Short:  We first create Dockerfile and then push it to Github or Bitbucket. After authenticating Github account with Dockerhub we choose repository on Dockerhub from Github that contains the Dockerfile. After that the Dockerfile triggers build which in result creates Docker image getting ready to pull.

I will share an example of making a CentOS image having Apache httpd server pre-installed.

First we need to create a Dockerfile which can be viewed here.

FROM centos:centos6
MAINTAINER trishnag <trishnaguha17@gmail.com>
RUN yum -y update; yum clean all
RUN yum -y install httpd
RUN echo "This is our new Apache Test Site" >> /var/www/html/index.html
EXPOSE 80
RUN echo "/sbin/service httpd start" >> /root/.bashrc 

Then push the Dockerfile to Github. I have created a repository named CentOS6-Apache and pushed the Dockerfile to it. The repository can be found here.

After doing so

  • Go to DockerHub and Settings —> Linked Account —> Link your Github account.
  • Create —> Create Automated Build —> Create Auto-build Github.
  • Select the repository that contains Dockerfile of your application.
  • Create followed by the Automating build process.

After the image is built successfully you will be able to see Success in Build Details which indicates that the image is successfully built. The Image is now live on docker hub https://hub.docker.com/r/trishnag/centos6-apache.

Now we have to test the image to make sure whether Apache httpd server is actually pre-installed or not.

docker pull trishnag/centos6-apache  #Pull the image from Dockerhub
docker run -t -i -d trishnag/centos-apache /bin/bash #Run the container as daemon
docker ps #Get the name of the container

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8bcd1199bb8f trishnag/centos6-apache "/bin/bash" 2 minutes ago Up 2 minutes 80/tcp jovial_cray

docker logs jovial_cray #To see What is happening on bash of container.Gives httpd status
docker inspect jovial_cray | grep IPAddress #Shows the IPAddress
curl 172.17.0.1 #curl the IPAddress we got
This is our new Apache Test Site

We just have got the text that we echoed out to index.html. Hence we can finally come to conclusion that Apache httpd server has already been pre-installed in the image.

Now even if we commit changes to the Dockerfile on Github we really don’t have to worry about the image. The build process of Docker image starts automatically once any changes are committed to Dockerfile. When we pull the newly built image and run container we will be able to find those changes.

Automating build process really makes our life easier.

Resources:

 

Handling Deadlock

This post is all about handling deadlock and what I did to avoid deadlock. This is the first time I have worked on handling deadlock. I have chosen an issue from fedora-bodhi which was about handling deadlock. The error was ”Deadlock found when trying to get lock; try restarting transaction” which is usually common when it comes to database.

Deadlock is an unwanted situation that arises when a process waits for indefinite time for a resource that is held by another process. So there is some way to avoid deadlock or handle deadlock. The transaction involved with the deadlock can either be rolled back or restarted.

try:
    Perform table transaction
    break
except:
    Catch the error
    Try again to perform table transaction

Lmacken guided me with some hints to handle Bugzilla deadlock. Since it was bugzilla server side issue it was not simple for me to reproduce the bug. But it was quite easy to handle bugzilla server-side deadlock. The deadlock was arising when an user was going to add comment to a bug in Bugzilla. So it was definitely a server side issue but could be handled with some little trick in bodhiLmacken told me to catch the fault triggered by the call of adding comment and try again. It was an xmlrpclib.Fault exception. Thus the technique to handle deadlock is the same as declared above. And I was finally done with my patch. Thanks to Lmacken for helping me to solve the bug and understand the technique :).

Hence I learnt how to handle deadlock and my pull request was merged 🙂

AutoShare: Automates sharing of video on Facebook when uploaded on YouTube

AutoShare automates sharing of video url with its title on facebook wall when the video is uploaded on YouTube. It took me about a couple of days to make AutoShare finally work. I have used Python 3. You might be wondering that how the idea came to my mind. Let me share the story with you 😉

My mom has her own channel on YouTube. So she has to share the videos on social networking platform like facebook and others everytime she uploads them on Youtube. So I thought of a way if I can automate the task! Life is awesome when we can automate almost everything, atleast for me 😛 so why not this 😉 .

AutoShare is really easy to use. You just have to get your PlaylistId from Youtube channel list and your google API key from console developer google. Check Youtube-settings for AutoShare to know in details. Then obtain your Access token from facebook api explorer. Check Facebook-settings for AutoShare to know in details. After obtaining the google API key, youtube PlaylistId and facebook Access token, fill them in settings.py. Make sure you have requests module installed and run the app with

python3 autoshare.py

Check the installation procedure here.

I have used requests and json modules to make AutoShare work. Using get method I have retrieved the data regarding the latest video uploaded and generated the url and title of the video which you will find in youtube_url_generator.py. You will find the code regarding posting in autoshare.py. I have retrieved the posts of facebook wall and run a check to make sure whether the post has already been shared or not. This will prevent you making duplicate posts on facebook wall. And if there is no such post on your facebook wall AutoShare posts the video title along with the url on your wall.

So this way we can easily automate the sharing of video on facebook wall when uploaded on Youtube. Test cases for the application are in progress. Then I’ll cut a new release :). Please do let me know if you get any issue or have any new idea. Feel free to open issues here and you can also come up with pull requests for modifications. Thank you 🙂

Resources:

 

Tell-me-more

I am pretty sure that you are wondering why such a title!

Tell-me-more is a command line tool built using Python3 that provides information about the linux user and the system. The tool displays the users of the system, the current user who is logged in the system, and the memory information.

To run the tool follow README .

I have used regex-regular expression in Python3 for pattern searching. I have taken help of re module for pattern searching, argparse module for taking command line argument.

Check the source code in github to stay updated.

Finds the users


def bash_users():
    with open('/etc/passwd', 'r') as fobj:
        f = fobj.readlines()
        pattern = '/bash'

        for i in f:
            m = re.search(pattern, i) #searches if the given pattern is in i
            if m:
                print("%s" % i.split(':')[4])
            else:
                pass


Finds details of the memory of your system



def memory_details():
    with open('/proc/meminfo', 'r') as fobj:
        f = fobj.readlines()
        pattern = '^Mem' #All the strings starting with Mem

        for i in f:
            m = re.search(pattern, i) #searches if the pattern is in i
            if m:
                print("%s: %d MB" % (i.split(':')[0], int(i.split()[1])/1024))  #Displays the memory in unit MB
            else:
                pass

See howitworks for a demo of Tell-me-more. Fork me on github to contribute for this project. Thank you 🙂

Finally the irc bot can save the channel’s logs

Finally batuli can save the channel’s logs. I can still remember the very first day when I and sanketdg joined the irc channel ##testbot to work for irc bot batuli and I asked sanketdg “Won’t batuli save the logs of the channel as batul does for dgplug?” , sanketdg replied to me “Make it save the logs if you can!”.

I couldn’t ever imagined that making batuli save the logs of the channel could be done in so simple way. Ah yes, I spent my days studying about logging module and twisted log module. And i took help of twisted log module to make batuli save the channel’s log and sent a pull request. But sanketdg told me to make it simple and he suggested me to use logging module, yes he was pretty correct. The script was looking pretty complex. Hence I started working with logging module. On the next morning suddenly an idea came to my mind. I asked myself that why I am using these logging modules to save the channel’s log, I also have other options in Python . Because, Python is such a programming language with what you can do just anything. Coding in Python is nothing but playing with it 😉 . And what I did is just append the message of user to a file. There couldn’t be any simpler way to make your bot save the logs. Let me share it with you 🙂

with open(args.filename,'a') as fobj:
    fobj.write('\n' + time.strftime('[%d-%m-%Y %H:%M:%S]') + \
            ' <' + user + '> ' + msg)

To take command line argument, I have used argparse module instead to sys module to make the command line interface more interactive with user and it was an idea of sanketdg  🙂

So I am really happy that batuli now saves the logs of ##testbot. Join our channel ##testbot on irc.freenode . Fork us on github . Follow README section to know how to use batuli , just give it a try and I am sure that you will love it 😉

Calci

I have built Calci, a command line calculator for linux that performs basic operations like addition, subtraction, division and multiplication. I have used Python3 to build this command line tool. You will find that I had already made a blog post on Command line calculator earlier. Now this new post comes with some edits of the earlier one and another important thing is that I have made this open source .

To run Calci follow the steps given below:

  • git clone https://github.com/trishnaguha/Calci.git
  • python3 main.py followed by arguments

Since it is a command line tool you’ll need to provide arguments after python3 main.py . Type python3 main.py -h or python3 main.py –help to know this in details.

you can also fork Calci on github and send pull requests to add extra features or edit the existing features. You are also invited to open issue if you find any.

I am really excited about this project, Looking forward to turn this into a scientific one. 🙂

Finish what you started, don’t give up

“Finish what you started, don’t give up” one of the most important motto of Open Source Project. This is not applied for just only Open Source but also for everything.

I have learnt something great from sanketdg in dgplug, can be termed as life lesson, i.e, “Completing the task that I have started or committed at any cost with out giving it up. “

I sent a pull request to pym in GitHub on some typo. On Wednesday kushal suddenly told to me to resolve conflict of the pull request that I have sent. My branch was having conflict , that’s why kushal was not able to merge the pull request with his master branch. And it took almost three hours to resolve the conflict.

Actually what I did was I fetched the upstream from git and rebased it  with my master branch. But the branch from where I sent the pr I forgot to it sync with master branch, thus my branch was totally messed up. sanketdg who knows git very well went on telling me the steps to resolve the problem. Two hours passed, still the conflict was not resolved. I told him that I better close the pr that I have made and open a new one. He told “nah. we better resolve this”. Every time I told sanketdg the same, he replied me the same. And finally with his help I solved the conflict. I learnt that it is necessary to always rebase your local branches with the master branch changes.

This is what we face in real time problems. There is no way to move away when you are having an issue/problem, we will have to solve it at any cost. In the same way we,who are now practicing can’t take a back because of being unable to solve a problem. What we have started, we have to finish it. Giving up is not an option.

Thank you 🙂