Test Video Maker

Recently I've been working on a very simple JS video slideshow. The only thing technically interesting about it is the order in which it plays back videos. I have some rough unit tests, but to demo that it's working correctly to the client, sometimes its nice to show it actually running the way they've asked you to make it.

Generally this involves me hunting around the internet for webm videos, and then trying to remember which order I've uploaded them while we wait for the slideshow to move through its queue and insert new videos into the correct spot.

I don't know why it's taken me so long, but finally I knocked together this little command line utility so I can make webm videos that have some text in them so I can easily label them with ordinals.

I'll add support for other formats and sizes, but it's easy enough to grab the code and alter it to get this working for you.

test-video-maker

2013-03-09
Some Drawings of Brooklyn

2013-02-24

How to Post a Photo to Facebook with Python

I don't know why you'd want to add content to facebook.com, but sometimes people ask you to do this for them. Adding photos can be a little tricky.

I was recently trying to do this in a Flask application using Flask OAuth. Unfortunately, if you pass a file along with the data to your remote method call, Facebook throws a dreaded "requires upload file" message. After doing a little research, it seems the problem has to do with the the file data getting signed by the oauth2 library. All the other data should be signed, then the file data needs to added, unsigned, afterwards.

This means patching the libraries to do what you want. I started going down that road and then I remembered I saw a PHP example where the access token was simply passed along as a querystring paramater and the file was uploaded normally with an HTML form.

Turns out, this works wonderfully and you don't need to jump through all the oauth library hoops.

Here's some code using Flask OAuth to get the access token and Requests to post the data.

oauth = OAuth()

facebook = oauth.remote_app('facebook',
    base_url='https://graph.facebook.com/',
    request_token_url=None,
    access_token_url='/oauth/access_token',
    authorize_url='https://www.facebook.com/dialog/oauth',
    consumer_key="GET YOUR OWN KEY",
    consumer_secret="GET YOUR OWN SECRET",
    request_token_params={'scope': 'publish_stream, photo_upload'},
) 

def get_image_path(filename):
    "Implement this to return an absolute path to the image. Maybe do something to avoid security issues."
    pass

@app.route('/fb/auth/post/<string:filename>')
def post_to_fb_auth(filename):
    callback_url = 'http://localhost' + url_for('fb_authorized', filename = filename)
    return facebook.authorize(callback=callback_url)

@app.route('/fb/authorized/post/<string:filename>/')
@facebook.authorized_handler
def fb_authorized(resp, filename):
    access_token = resp['access_token']
    image = get_image_path(filename)
    files = { 'source': open(image,'rb'), }
    url = 'https://graph.facebook.com/me/photos?access_token=%s' % access_token
    r = requests.post(url, files=files)
    return redirect(url_for("success"))

Just a few things to note: you have to add your callback url to your Facebook app's configuration. This the field called "Website with Facebook Login". In my example, I'm just using localhost, but you can add a port if you're not on 80.

You'll also need to make sure you have the publish_stream and photo_upload permissions set up in your application.

2013-02-24

My Favorite Blimp Photo

It's got everything you want in a blimp photo.

2013-02-20
Very Useful Ubuntu Reminder Alias

Add
1
alias remind='echo "notify-send Reminder" | at $1'
to your .bashrc, then:
1
remind 21:32
and you'll get a notification at that time. Very useful for setting time limits on tasks.

2013-02-20
Getting 'top of the day' epoch in Python

Python's time, date and datetime modules are powerful, but I find I nearly always have to go to the docs to do even basic tasks.

I needed to get the epoch time for Midnight of a day. It took me way to long to figure how to do this, so I'm stowing this away here for next time.
1 2 3
import calendar import datetime calendar.timegm(datetime.date(2012,9,1).timetuple())
2012-09-01
Mouse Hit-Testing on a Canvas Using a Separate Buffer's Pixel Data.

While unsuitable for such a simple example, this is a demonstration of using a seperate buffer for quick and dirty mouse selection in a canvas element.

Clickable items, here circles, are given an int id (random in this example, but these could increment from 0, etc). This id is converted to a hex color which is used as a key in a dictionary, pointing to the associated Clickable object.

When the mouse is pressed, the clickables are rendered to a seperate canvas with the same dimensions, but instead of using their ordinary display color as a fill color, we use the color that derives from the clickable's id. The color is then read off of the buffer and used to look up the object from the clickables dictionary.

On first glance this technique seems overly complicated, particulary for circles where simply checking if the mouse falls into the radius of the clickable objects would be faster and easier, but it has a number of advantages: It works with arbitrarily shaped objects, it would work even if you were faking a 3d scene, and you don't have to do complicated math to translate your mouse coords if you're positioning your objects by transforming the canvas geometry with translation, scale and rotation calls. You can also easily change item depth by sorting the objects so objects can occlude one another in whatever order you wish.

There are some caveats. Canvas doesn't support aliased rendering, so you can get mis-attributed hits if you happened to click the edge of a shape whose pixel color happens to match the id color of another object. It's very dificult to select more than one object at a time. You also need to make sure your canvas background color is not a viable id.

2012-08-12

Installing Django on mod_wsgi, in a virtualenv behind Nginx on Ubuntu

These instructions will help you walk through setting up Django in one of the preferred deployments: We'll use Nginx as a proxy to pass off requests to Django. One of the benefits here is that you can set up other sites on your server running on various technologies and nginx will just pass the requests to the correct handler. This way, you can have some static sites, maybe a couchdb instance, a rails app, a Django app etc, all on the same server, which should be fine for low traffic sites. Django will run on Apache under modwsgi and live inside a virtualenv so you can install libraries or upgrade your Python wihout messing with your system Python.

I'm using a Linode instance with Ubuntu 10, but the instructions should be helpful for any Linux. One last thing, I'm not an ops guy. If you see something wrong, please let me know.

Before beginning, I'd just go read through the docs on deploying Django on mod_wsgi and running mod_wsgi on virtualenv. Both are pretty good. If you have some experience configuring a server, they're all you need. If you don't, having them in your head will be handy as you work through the following and they'll be crucial during trouble shooting. Either way, let's get started..

First, install virtualenv with easy_install. Create a directory in /usr/local/ called pythonenv and change into it, so you're in:

1	/usr/local/pythonenv

Then, create a clear virtualenv by running the command:

1	virtualenv --no-site-packages BASELINE

Create a user and under that create a directory sites/ and cd into it, so your pwd is something like this:

1	/home/someuser/sites/

In this folder, create a new virtualenv; this is the virtualenv that you'll install Django into.

1	virtualenv --no-site-packages example.com.env

Now if you ls example.com.env you should see:

1	bin include site

Create two more directories, site and src.

1	mkdir example.com.env/site; mkdir example.com.env/src;

Checkout Django into the src directory

svn co http://code.djangoproject.com/svn/django/trunk/ example.com.env/src/django-trunk

Then add django to the environment's path. You can do this with a symbolic link, or a .pth file. Your choice. Here's the .pth file creation:

echo $PWD/example.com.env/src/django-trunk >
example.com.env/lib/python2.6/site-packages/django.pth

(Note, check your python version above). If you want, you can set up django-admin on your shell path. Complete details on this can be found on the Django site.

At this point you should be able to activate your virtualenv and import django.

source example.com.env/bin/activate
python
>>> import django

Now, create or move your Django application code into the site directory you created a few steps earlier. You'll need to create a wsgi file if you don't have one already and add it somewhere in this folder. For help creating the WSGI file, follow the docs on djangoproject.com, but skip the Apache configuration part at the beginning. Just make the wsgi file and add it to a directory in your Django project. You'll also have to add the following to the top:

import site
site.addsitedir('/home/someuser/sites/example.com.env/lib/python2.6/site-packages')

Change the directory above to match the site-packages directory in your new virtualenv.

I personally set up two versions of my Django settings under a common "configs" folder with separate subdirectories for wsgi and settings files which correspond to my development and production environments. This is a technique I think I got from Simon Willison, but you can keep it anywhere in your project. Most people create an "apache" directory and stick it there. Just make sure all the configurations coming up point to it. Now follow the instructions to set up Apache2. but don't follow the virtual host configuration. When you get there, instead install modwsgi:

apt-get update
apt-get upgrade
apt-get install python-setuptools libapache2-mod-wsgi

Then configure your virtual host with the following instructions. First, put the following in your ports.conf file.

1	NameVirtualHost *:8080

Or whatever port you want to run Django on. Remember Nginx will be listening on 80 and passing off requests to Apache on some other port.

Then go to your apache conf folder (probably /etc/apache2/) and inside the site-available folder, make a file called example.com or whatever your domain is. Inside, put something like the following:

<VirtualHost *:8080>
    ServerAdmin webmaster@example.com
    ServerName  www.example.com
    ServerAlias example.com

    <Directory /home/someuser/example.com/site/configs/prod/ >
            Order deny,allow
            Allow from all
    </Directory>

    WSGIScriptAlias / /home/someuser/sites/example.com.env/site/configs/prod/apache.wsgi
    WSGIDaemonProcess  example.com user=www-data group=www-data threads=25
    WSGIProcessGroup example.com

    ErrorLog  /var/www/example.com/logs/error.log
    CustomLog /var/www/example.com/logs/access.log combined
</VirtualHost>

The important thing in this configuration is that the Directory directive points to the folder you put your Django wsgi file and the WSGIScriptAlias points directly to it. While you're at it, you can place your error logs whereever you want as well. The above might not be a great place for them. Make sure the directories exist.

Ok, now enable this site by running a2ensite

1	sudo a2ensite example.com

This creates symlink from your available sites to your enabled sites.

Also, edit your main apache2.conf file in /etc/apache2/ and at the bottom of the file add:

1	WSGIPythonHome /usr/local/pythonenv/BASELINE

Now you've installed a virtualenv with Django and Apache with mod_wsgi, so we're getting close.

If you haven't already, you’ll need to create a DNS entry in the Linode manager for your domain and point your domain to the Linode nameservers.

Edit your /etc/hosts file so it has a line like this.

1	127.0.0.1 localhost.localdomain localhost

You can add your domain to this too. Add it as 127.0.0.1 or your external IP name.

Make up some name for your server and make it your machine's hostname. For our purposes I'm going to use 'paintbrush'.

echo "paintbrush" > /etc/hostname
hostname -F /etc/hostname

Now install Nginx using the linode instructions

Then edit your nginx config file, which is probably at:

1	/opt/nginx/conf/nginx.conf

Nginx has a straightforward configuration syntax you should be able to figure out just by reading the examples. Follow Linode's guide. But the following is what you want...

server {
    listen       80;
    server_name  *.example.com;
    access_log   logs/example.com.access.log;
    error_log   logs/example.com.error.log;

    location / {
        proxy_pass http://example.com:8080;
        include /opt/nginx/conf/proxy.conf;
    }
}

Finally, you can add location configurations to the above to server your static media. Something like this should work, but again, refer to the Nginx documentation.

location /media/  {
   root    /home/someuser/example.com.env/site/static/
}

Now, retstart Apache and Nginx

sudo /etc/init.d/apache2 restart
sudo /etc/init.d/nginx restart

Fire up a browser and go to your domain. Guess what? it didn't work. This is a complicated process and you're bound to mess something up. The key here is that you need to check your logs. Tail the nginx and apache error logs that you configured above and fix each problem until you have this working.

If you see import errors, that means your Python path isn't correct. Make sure you really understand it, fix it and don't forget to restart Apache when you're testing fixes.

2011-05-11

Swinging Effect with CSS3 3D Transforms

I've been looking at the 3D transforms that the latest webkits have included. Here's a quick proof of concept for something I'm working on. Should work in newish releases of Chrome or Safari.

2011-01-21
Py2Exe and Multiprocessing

So I never forget: If you need to use multiprocessing within an app bundled with Py2Exe, you need to use multiprocessing.freeze_support().

2010-08-06

Emailing Content to Django with Webfaction's mail2script

Update: in recent versions of Django, its much better to use custom management commands

Webfaction, my shared-hosting provider of choice, has a nice feature that allows you to pipe emails to a script. This is useful for all sorts of things, but I decided to try it out by building a way for a friend to email photos to the front page of his website, which is built with Django.

To start, create a python script (I called mine EmailPhotoUpload.py). You can place it anywhere really, but I placed mine right in the django app that had the models I would be working with. Make sure the script starts with

1	/usr/local/bin/python2.5

or whatever Python you're using with webfaction. Also, make sure it's executable:

1	chmod +x EmailPhotoUpload.py

When mail2script executes the script, it doesn't execute with knowledge about your Django project, so you'll need to set the DJANGO_SETTINGS_MODULE environment variable and place your project on your Python path if its not already. Something like this works:

PROJECT_ROOT = '/home/webfactionuser/webapps/django/yourproject/'
DJANGO_SETTINGS_MODULE = "settings"

def setup_django():
    sys.path.append(PROJECT_ROOT)
    keys = os.environ.keys()
    from re import search
    for key in keys:
        if not search("DJANGO_SETTINGS_MODULE", key):
        os.environ["DJANGO_SETTINGS_MODULE"] = DJANGO_SETTINGS_MODULE     

setup_django()

Fix the project root variable to point at your project.

Once you've done that, you can import the models you want to manipulate. I have a model called 'Photo' in the 'photo' app. So I import it like so:

from photo.models import Photo

The email is accessible at sys.stdin, which is a file-like object. Python's email module has a method that takes a file and makes a Message object, so this works out nicely:

msg = email.message_from_file(sys.stdin)

Now you can refer to the email module documentation to get your data. I started by writing a few helper functions to get the data I want out of email.

def get_text(msg):
    "Looks for a part of the message with a text/plain mime type and returns it"
    text = ""
    if msg.is_multipart():
        for part in msg.get_payload():
            if part.get_content_type() == 'text/plain':
                text = part.get_payload()
    else:
        text = msg.get_payload()
    return text

def get_jpegs(msg):
    "Looks for any part of the message with image/jpeg mime type and returns a list."
    jpegs = []
    if msg.is_multipart():
        for part in msg.get_payload():
            if part.get_content_type() == 'image/jpeg':
                data = part.get_payload()
                tempjpeg = tempfile.NamedTemporaryFile('w+b', -1)
                tempjpeg.write(base64.b64decode(data))
                jpegs.append(tempjpeg)
    return jpegs

A few things to note: I'm using Python's tempfile module to hold the jpegs until I can save them in my model. Django has its own NamedTemporaryFile class, but it's for Windows compatibility and since this is a linux environment, we can use the standard library's. All we need to do is take the payload from the message part, base64 decode it and write it into a file. If you expect large files, or many, you'll eat up memory here and you'll have to do something more sophisticated.

The Photo model I'm working with looks something like this:

class Photo(models.Model):
    title = models.CharField("Photo Title", max_length=255)
    original = models.ImageField("Image", upload_to="img/uploaded/original")
    resized = models.ImageField(upload_to="img/uploaded/resized")
    credit = models.CharField("Credit", max_length=255, blank=True)
    caption = models.TextField("Description", blank=True)

I'm going to use the From field to populate the photo credit, the text portion of the email to populate the caption, the subject line of the email to populate the photo's title.

sender = re.sub("\s<(.+)>(\s*)$", "", get_header(msg, 'From'))
subject = get_header(msg, 'Subject')
text = get_text(msg) 
jpegs = get_jpegs(msg)
basename = re.sub(r"[\\/ \(\):?;,]", "_", subject)

For any of the header information, just call get_header, pass in the name of the email header and then perform any clean up on the result. For instance, I'm stripping the email address portion from the "From" header. Now just iterate through your list of jpegs and make a Photo object for each:

for index, jpeg in enumerate(jpegs):
    photo = Photo()
    photo.title = subject
    photo.caption = text
    photo.credit = sender
    filename = "".join([basename, str(index), '.jpg'])
    photo.original.save(filename, File(jpeg))
    photo.promote = True
    photo.save()
    jpeg.close()

Remember, jpegs is a list of temporary files; be sure to close them so the os cleans them up.

Now log in to webfaction, create an email address and put the absolute path to this script in the target area.

Please note, this isn't very secure. Anyone with the address can post content, and if you're not cleaning it, they may be able to do worse--never trust user input.

I plan on releasing a cleaner, more reusale classed-based version with some added features and exception handling, but for now I'll post the full text of what I describe above so you can see everything in context. Remember, you can test your script by downloading the raw email text and redirecting it to your script at the command line like this:

1	./EmailPhotoUpload.py < test-msg.txt

Good luck!

:::/usr/bin/env python

import os
import re
import sys
import email
import base64
import tempfile
from django.core.files import File

PROJECT_ROOT = '/home/justin/Documents/sites/workshop/workshop/'
DJANGO_SETTINGS_MODULE = "settings" 

def setup_django():
    sys.path.append(PROJECT_ROOT)
    keys = os.environ.keys()
    from re import search
    for key in keys:
        if not search("DJANGO_SETTINGS_MODULE", key):
            os.environ["DJANGO_SETTINGS_MODULE"] = DJANGO_SETTINGS_MODULE  

def get_text(msg):
    text = ""
    if msg.is_multipart():
        for part in msg.get_payload():
            if part.get_content_type() == 'text/plain':
                text = part.get_payload()
    return text

def get_jpegs(msg):
    jpegs = []
    if msg.is_multipart():
        for part in msg.get_payload():
            if part.get_content_type() == 'image/jpeg':
                data = part.get_payload()
                tempjpeg = tempfile.NamedTemporaryFile('w+b', -1)
                tempjpeg.write(base64.b64decode(data))
                jpegs.append(tempjpeg)
    return jpegs

# Start
setup_django()
from photo.models import Photo
msg = email.message_from_file(sys.stdin)

address = msg.get('Return-Path', '').strip("<>")
sender = re.sub("\s<(.+)>(\s*)$", "", msg.get('From', 'Unknown'))
subject = msg.get('Subject', 'Untitled')
text = get_text(msg) 
jpegs = get_jpegs(msg)
basename = re.sub(r"[\\/ \(\):?;,]", "_", subject)

for index, jpeg in enumerate(jpegs):
    photo = Photo()
    photo.title = subject
    photo.caption = text
    photo.credit = sender
    filename = "".join([basename, str(index), '.jpg'])
    photo.original.save(filename, File(jpeg))
    photo.promote = True
    photo.save()

for jpeg in jpegs:
    jpeg.close()

2009-12-21

Justin Donato

Test Video Maker

Some Drawings of Brooklyn

How to Post a Photo to Facebook with Python

My Favorite Blimp Photo

Very Useful Ubuntu Reminder Alias

Getting 'top of the day' epoch in Python

Mouse Hit-Testing on a Canvas Using a Separate Buffer's Pixel Data.

Installing Django on mod_wsgi, in a virtualenv behind Nginx on Ubuntu

Swinging Effect with CSS3 3D Transforms

Py2Exe and Multiprocessing

Emailing Content to Django with Webfaction's mail2script