Remember the sabbath day

remember_the_sabbath

I made a small Python script for checking how many Git commits have been done during the weekend days. It is a simple idea. Only mark the days when commits have been done. Don’t think too much about number of commits or how many changes have been done. It is up to us to interpret the graph, but you apply it on a project which has been going on for a very long time you can kind of read some things out of it.

Look at the picture above. Graph in the top left corner is from a commercial project. Clearly, nobody cares about it over the weekend. Percentage of activity is around 20%, which means that work has been spread proportionally.

Graph in the bottom right corner is from Django repository. More or less this is how all big Open Source projects look like with lot of participants. Work evenly spread over the entire week.

You can find out more here:
https://github.com/aerkalov/sabbath

Importer 3000!

importer3000

Importer 3000 is here! What is it? It could be the most important moment in the history of Booktype! It could be, but in reality it is just a basic upload form I did for testing new ebook library.

importer_basic

And yes, it could easily look like this but would anyone believe me I have spend more then 5 minutes on this task? Can that simple form explain why am I so excited about new ebook library? Probably not and that is why we have Google fonts and shiny colors! If you are trying to present couple of months of research and work you have spend on backend code, try to use them. It can not hurt.

github

 

Booktype developer’s review: The Django Circus comes to

Original post is here:
https://www.sourcefabric.org/en/community/blog/1878/Booktype-developer%27s-review-The-Django-Circus-comes-to-town!.htm

If I only knew how awesome the Django community is, I would have started attending DjangoCons way earlier. This year the European DjangoCon was held in Warsaw from the 15th to the 19th of May and believe it or not, it was in a Circus tent! It was organised by members of the local Django community and they have really raised the bar for the next DjangoCon in France. Because Django is a central programming language for Booktype I decided to go and check the action from this community of developers and aside from learning a lot of cool new tricks I also had an amazing time!

Fun in the sun

The first three days of the conference were organised at the horse track, next to a weirdly big fountain and an old swimming pool. There was only one track at the time and that proved to be more than enough. The tent was open to the outside providing good airflow and easy access to the lectures. It was outfitted with power, WiFI, projectors, tables for those who can not escape their work and even a refrigerator filled with cold water and drinks. During the day people could easily grab coffee, fruit, sandwiches, homemade energy bars, ice-cream and popcorn in the smaller tent. The organisers provided us with hammocks, deckchairs, bean bags, blankets, frisbees, badminton rackets and many other toys. Many times it just felt like we were at a music festival and not a developers conference.

Django experts visit the festival

But it was not just about the food and frisbee. The event was packed with interesting people and projects, and this is what conferences are all about, meeting new people. Russell Keith-Magee (President of the Django Software Foundation), Andrew Godwin (Django and South), Tom Christie (Django REST Framework), Kenneth Reitz (Requests and Python Software Foundation), Steve Holden (Python Software Foundation), Zed Shaw and Aymeric Augustin (Django) were all present, just to name a few.

You could feel the spirit of open source and the Django community at the festival. Everyone was more than friendly and very approachable. I guess all the stories about how the Django and Python have one of the friendliest communities are true after all! The best place to see this was during the last two days of the conference. The coding sprint was organised at the Gamma Factory and the first day it was attended by more than 200 people. Instead of junk food the organisers provided us with more healthier meals. But enough about the food. Members of the Core Django team and developers who have already contributed to Django were more than helpful and patient with the newbies and people who just wanted to help with coding, documentation, testing or sometimes just organising old tickets in the system.

It was a very positive and enriching experience. I encourage you to think about participating at DjangoCon next year or just visit one of the local Django/Python gatherings in your country. You will not know how stimulating it can be until you try it!

As they say, a picture tells a thousand words! Check out the video from the festival.

E-Book Web Reader ~ Rheya ~

info

I had a small coding sprint over the weekend. It would probably be better if I played Minecraft entire weekend but somehow i couldn’t stay still until I put my thoughts into code….

The code is not really representative, I just wanted to check the feeling of using this new Web interface for the books. I have two not fully functional interfaces: desktop and mobile one. I am aware it does not work on Android (the footer), the inner frame do not resize as they should and some buttons do not work. Like I said… just a proof of concept for now. But a proof of concept which has support for keyboard shortcuts and you can change color/text theme. What more do you need?!

mobile_font

But when it is finished… ohhhhh when it is finished! It will work in offline mode, it will support bookmarks, sharing, annotations and many many more things. It can read epub2/epub3 books for now but I will integrate it with Booktype online books when I get some free time.

You can check the code here:
* https://github.com/aerkalov/ebook-web-reader/wiki

Demo books:
* http://www.binarni.net/projects/rheya/metropolis/ (Desktop version)
* http://www.binarni.net/projects/rheya/the-binary-chronicles/ (Desktop version)
* http://www.binarni.net/projects/rheya/metropolis/mobile.html (iPad version)
* http://www.binarni.net/projects/rheya/the-binary-chronicles/mobile.html (iPad version)

Soooo…. what do you think? Would this be a good interface for online/offline books?

Booktype-scrolls: Ubuntu installation

First in a series of screencasts – How to use Booktype-scrolls script to install Booktype on Ubuntu.

Script is smart enough to figure out what needs to be installed on Ubuntu, Debian, CentOS or Mac OS X. You will be informed about every step and nothing will be installed on the system without your confirmation or knowledge. At the moment script works with Sqlite and builtin Python server. That will be changed in next version but for now this is more then enough if you had problems installing it for yourself. You can read more about the script in my previous post – http://www.binarni.net/2012/07/booktype-easy-install/.

For more information check out official installation docs – https://wiki.sourcefabric.org/display/Booktype/Installing.

Styling your book on the Web

As you know, you can use Booktype to publish your books into different formats but you can also view them on the web. In this blog post I will try to show how that web view can be customized with simple template modification. By default Booktype comes with predefined templates but they can all be easily customized to your own needs (templates can also be multilingual).

Let us imagine we are publishing shop “Awesome Sun” who is interested in publishing books about solar power. Booktype project has been installed in /var/www/awesomesun/ directory on our web hosting server. We are not so happy with the default look and feel so we decided to change it. Where should our web designers look at? In the case of “Awesome Sun” template files should be placed in /var/www/awesomesun/templates/ directory while all static files (CSS files, JavaScript files, images, …) should be placed in /var/www/awesomesun/static/ directory. Fairly simple for now. Before doing any work it would be wise to check documentation how to work with Django templates. As you could guess, examples in this blog post are oversimplified.

By default, to present books on the web Booktype comes with Django application called “Reader” and we will customize templates for that application. Original templates for this application are placed in $BOOKTYPESOURCE/lib/booki/reader/templates/ directory and our customized templates will be placed in /var/www/awesomesun/templates/reader/ directory. All you have to do is either copy original template files into awesomesun project and modify them or write new templates from scratch.

What kind of modifications are we making? As you can see on the screenshots bellow we want to have very simple book view, we want our “Table of contents” to be always visible while we scroll the page and we want our header to be fixed on top with name of the book and chapter we are currently reading.

To make it work I created new base template called new_base.html and placed it in /var/www/awesomesun/templates/ directory. This is a base template which other templates for “Reader” application should include. This template just includes jQuery library and creates place holders for future content.


{% load i18n messaging_tags booki_tags %}



{% booki_site_favicon %}
{% booki_site_metadata %}

{% block header %}
{% endblock %}

{% block content %}
{% endblock %}



Then I had to create /var/www/awesomesun/templates/reader/book_chapter.html template file. This is where I include additional JavaScript library, CSS file, display “Table of contents” and content of chapter. Besides HTML and CSS there are just two important things to look at. Chapter content is inserted with {% booki_format content %} and “Table of contents” is inside of variable chapters. It all depends how you want to show your “Table of Contents” but here I am using “for” loop to construct unordered list with chapter titles. Everything else is just CSS styling.


{% extends "new_base.html" %}
{% load i18n booki_tags %}

{% block header %}
{% blocktrans with book.title as booktitle %}/book: {{ booktitle }}{% endblocktrans %}

{% endblock %}

{% block content %}

{% booki_format content %}

{% endblock %}

Essentialy this is how you customize Booktype templates. I have compiled all needed files (with CSS and additional JavaScript) from this blog post here. Please download it if you are interested in more details.

Booktype Easy Install

A lot of people seems to have problems installing Booktype. Until we have native packages for each distribution installing vanilla Booktype from git repository will be the way to go.

I love all of my users the same and that is why I decided to work on a small install script which would do most of the steps for them. This script should help you even when your distribution does not provide valid native packages (Old Django packages and etc…). Anyhoo, I have done some basic testing on – Ubuntu 10.04, Ubuntu 12.04, CentOS 6.3, Debian 6 and I have tried to avoid combination of outdated system packages and latest packages installed with PIP (because that is what Google told you to do). I am aware there are many places where this script could go wrong but i have tried to fix at least basic problems. For instance, if you are on CentOS it will warn you and inform how to install EPAL repository.

Like all “double click” methods this script will also install unwanted packages. To avoid conflicts with different versions of python modules it will create virtualenv environment and install only those Python packages (and their versions) you really need. It will install packages using python package tools and not system packages. For some of the packages it means compiling. Compiling means having development tools on the system. Be aware of that.

Created Booktype instance will use Sqlite to store data. Sqlite is great database but it does not support easy upgrades of schema. If you are serious about Booktype you might want to look at PostgreSQL. After all, this is version 0.0.1 and we only support built in python web server. To make it work with Apache you will need to manually activate virtualenv environment in wsgi script. In the future install script will do this for you.

Where to get it

For now, Booktype Easy Install script is part of “Booktype Scrolls” project and you can get it here:
https://github.com/aerkalov/booktype-scrolls/tree/master/scripts/install.

How to start it

Download the script and execute it. It will download and install required packages. When it is done it will tell you how to create user account and start Booktype. You don’t need to be root to start this install script but you do need sudo permissions in case it must install some system packages. You will need to confirm installation of new packages and you will be informed which commands are being executed in the background. Without your permission new packages will not be installed on the system. Feel free to analyze the script before you start it.

wget https://raw.github.com/aerkalov/booktype-scrolls/master/scripts/install/booktype_install.py 
python booktype_install.py

Use –help to get all available arguments:

python booktype_install.py --help
python booktype_install.py -p myproject
python booktype_install.py -o ubuntu -p book

What now

It needs testing and support for more platforms. Please report to me if you have problems with it. Will try to fix it together!

Booktype scrolls – Importing WordPress site!

So…you left your job and went traveling throughout Asia for 4 months.  In small cyber cafe, somewhere in south Pakistan, you created WordPress site because you were lazy to send emails to friends and family. Internets loved you and your pictures were on the front page of Reddit many many times. Who could forget that great article about food poisoning in China or that unfortunate misunderstanding with gentleman in Bangkok…

When you got back everyone was like – “You should write a book!” and you said to yourself  – “Why not… Half of the book is already written.”. This brings us to the subject of this post – importing WordPress site as a Booktype book.

Importing

There are couple of ways you can do it (this can be applied to other systems also):

  • Use some kind of API to remotely access data. For this you only need permission to access data.
  • Write your own plugin to export data from WordPress. Besides knowing how to write plugin you need permissions to install it on the remote server. You can use WordPress API to render the articles properly, something you can’t really do with other 2 methods.
  • Finally the method we are using in this post. Parsing XML export file created with WordPress export plugin.

Importing data can be complex process and each method has its good and bad points. You probably don’t want to have chapter for each post in WordPress because that could be hundred and hundred of chapters at the end. Maybe you want to choose which chapters you want and sometimes to combine couple of posts as one chapter. These are just some of the issues.

How it works

As it is visible on the picture above you should go to Tools/Export and choose what you want to export. Download the “Export File” to you computer. Export file is just extended RSS file and you can (and should) use it to export content of your site. Notice that attachments and images are not part of this export file, we will need to download them separately! Besides content is in “raw” format (before everything was rendered to HTML).

Like I said, export file is extended RSS file and for parsing we use this great library feedparser. Book name can be specified with arguments but for the default name we use title of the WordPress site. Booktype cares about two titles. One is the full title of the book and the other one is unique url name of the book (usually slugified version of full title).

from booki.utils.book import createBook

book = createBook(conf['user'],
                  conf['bookTitle'],
                  status = 'new',
                  bookURL=conf['bookTitleURL'])

As you can see we use function createBook. It takes user object as first argument (someone has to be owner of the book), full book title, default book status (books can have different statuses) and slugified title name. Export file provides us with the information about WordPress administrator and Post author. We could use that information to find or create Booktype user with specific info and set that person as Book owner.

Parsing posts

After this we just go through the list of all the posts in export file and ignore those who don’t have ‘wp_status’ set ‘publish’. Like i said, content of that chapter is in “raw” format. For instance empty lines in text are representing start of new chapter and etc. If we were using specific tags to format our code (for instance, to prettify the source code) we would not get nice and colorful HTML.

In this example we just do two modifications. One is to create paragraphs in text and the 2nd one is to put Chapter title inside of H2 tag. That is one of the Booktype requirements at the moment.

content = "\n".join(["< p >%s< /p >" % p for p in content.split('\n\n') if p.strip() != ''])
content = u'< h2 >%s< /h2 >%s' % (chapterTitle, content)

And yes… I have some extra spaces in tags P and H2 because i am too lazy to figure out how to escape it in code prettifier plus the code to make paragraphs is not perfect (but this is just an example).

Images

Now we parse Post content and search for the images. When we find one, we try to download it to our computer. Normally we wouldn’t need to do it, but Booktype really wants to have entire content of a book locally. When the image is successfully downloaded we save it as an attachment and we modify image location. All images must be placed inside of ‘static/’ directory. There are good reasons why it has to be relative path, but we will not talk about it now.

att = models.Attachment(book = book,
         version = book.version,
         status = stat)

f2 = File(StringIO(data)) f2.size = len(data) att.attachment.save(fileName, f2, save=True)
fileName = os.path.basename(att.attachment.path)
e.set('src', 'static/%s' % fileName)

Chapter

At the end we just create new Chapter. This is very basic way of creating new chapter because we are not leaving any traces in our logs and etc… We can leave that for some other example. Also notice we are not doing anything with links to other posts/chapters on the same site/book. We just ignore that for now.

chapter = models.Chapter(book = book,
                         version = book.version,
                         url_title = bookiSlugify(chapterTitle),
                         title = chapterTitle,
                         status = stat,
                         content = content,
                         created = now,
                         modified = now)

chapter.save()

Source

Look at the full source for WordPress importer in my Booktype Scrolls repository. Purpose of this code is to be more educational and less production ready.

As you can see technical part of importer is very simple. Thumblr has API for remote access and it would be fairly simple task to make Thumblr importer (or for any other CMS). The biggest problem is still how to make their HTML work nicely inside of Booktype. If author is using custom CSS on WordPress side then Booktype would be completely unaware of it. For instance – <img class=”alignnone size-full wp-image-591″ … It would be the same for different WordPress plugins, broken HTML, headings at wrong place, JavaScript plugins and etc.