Lean Startup, Software Testing and Python
 

Setup a remote development environment with Vagrant, Salt and Digitalocean

For years I have been doing web development on a lenovo S10 netbook via SSH. I like to be able to connect to it from anywhere in the world and resume my work by typing: tmux -2u attach. But it also has some issues:

  1. sometimes the connection is very slow in other countries because I have to SSH to my home router in Sweden.
  2. the laptop is getting older and the fan makes lots of noise.
  3. I cannot do system snapshot like a virtual machine.
  4. I cannot suspend/resume the system remotely like a virtual machine.

Since I have been using Vagrant and Salt for my web service, I figured I could setup a remote development environment using Salt. All of my repositories are at Github and Bitbucket anyway. As for the VPS,  I chose DigitalOcean because of their API, price and speed:)
Here is the github repository for my vagrant and salt files:

https://github.com/zheli/dev-environment

 

 

 

 


Strange import error in Django

Last night I started to have a string Import Error in my Django project in the development environment. It was working before and I haven’t changed any code yet.

The Symptoms

I have a utils folder at the root of my django project with the following structure:

utils
|
|-__init__.py
|-log.py

And the problem appears in my code when I use:

from utils import log as logging

It gives me Import Error in some files but not all the files. Also if I use the shell I can load this module without problem. This whole thing is very strange. I checked the PYTHONPATH and PATH variable, but nothing seems to be wrong. So I created a new modules called libs with the same content and renamed the module name in all the code that complains import error.

Solution

I decided to try to look into it again today to see if it can be fixed, but I couldn’t find anyone who has the same problem like me. Accidentally, I found an interesting comment:

+1 just had similar problem to OP and removing *.pyc resolved it so thanks. this seems to work nicelyalias rmpyc="find . -name "*.pyc" -exec rm -rf {} \;" to ‘clean’ a project

And it worked! So it is because of the *.pyc I got when I was working in other branches. I don’t know when Python will decide to recompile the *.pyc file (or perhaps git removed the .py file, but left the .pyc file, and the code wasn’t cleaned up). But this is definitely something you need can try next time when you have an “Import Error”


Some tips with Raspberry Pi development

Change the permission for i2c

It is very annoying that you have to run your code with sudo to be able to use I2C. You can change it by doing this.

First add your user to i2c group:
sudo adduser pi i2c
then change the udev rules, here I changed rules for all the i2c devices. You have to edit 60-i2c-tools.rules file in your /lib/udev/rules.d/ folder.
sudo nano /lib/udev/rules.d/60-i2c-tools.rules
And change the following from
KERNEL=="i2c-[0-9]*", GROUP="i2c", MODE="0660"
to
KERNEL=="i2c-[0-9]*", GROUP="i2c", MODE="0666"
Save the file, type in the command below to reload udev rules.
sudo udevadm control --reload-rules


SimCan Python extension for CrossControl CCSimTech API

Not sure if anyone needs this:)

Finally finished my first Python extension written in C language. It was a good practice. I have forgotten a lot about C, especially the pointers:) So use this library at your own risk!

The library is compiled in Visual Studio 2008 Express Edition so it only supports 32bit Python 2.7.x. It also doesn’t support unicode. Should be enough for CAN message I guess. Another important thing to know is I am not sure whether it works with CCSimTech non-developer license. I might test it later after summer.

This library implemented CanOpen, CanClose and CanReceive functions.

The file and document can be found here.

 


Visual Studio 2008 Express Project Settings for Python C Extension

Here are some notes for setting up VS 2008 Express C++ project for Python extension development. For detailed information, you should check out the official document.

Project type

Choose Visual C++->Win32->Win32 Project. The recommended project folder should be your Python source code folder.

Application settings

In the application wizard, choose Application type: DLL. And check the box for “Empty project”.

Project properties

configure_1

First choose Configuration->All Configurations. In Configuration Properties->Linker->Command Line, add this into Additional options textbox:

/export:init[yourPyMODINIT_FUNCname]

(for example, if the function name is initspam, this line should be /export:initspam)

Then choose Debug configuration. In Configuration Properties->C/C++->General, added the following two folder into Additional Include Directories (only if you created your project in the Python source code folder):

  • ..\..\Include
  • ..\..\PC

In Configuration Properties->Linker->General, change Output File to

$(OutDir)\$(ProjectName)_d.pyd

Additional Library Directories

..\..\PCbuild

Finally, in Properties->Linker->Input, add Additional Dependencies

python27_d.lib

That’s it, you are done! Release configuration is almost the same just don’t forget to remove _d for output file and additional dependencies.


Why you should learn some JavaScript as a test developer

As a test developer, I love making tools, and I like to share my tools. It is always great to hear about how cool your tools are or how much their lives have been made easier.

Most of my tools are written in Python. Simply because it is quick to prototype and there are lots of libraries around Internet.  But it has changed a bit after I started writing more JavaScript code in my current assignment. There are lots of new, exciting things happening around JS world now such as Meteor, AngularJS…

In the past it wasn’t easy to distribute my tools, one reason is that not everyone has Python on their machine. It is totally different with JS. If you have a browser, you can run my code. It can be remote on a server, it can also be in a local folder.

It is also super simple to have nice-looking UI in JS, because HTML/CSS is much easier to tweak. I am not saying I don’t like command-line interface, just sometimes it is more intuitive to have a cool graph. There are also many ready framework you can grab from the internet, including one of my favorite Twitter Bootstrap. Not to mention countless JS plugins to show off/manipulate your data.

angularjs

To get work done quick, you might also need a good JS framework. There are never lack of framework options in JS world. And that is also a problem. Personally, I prefer AngularJS. The learning curve is not steep (at least in the beginning:)), and the built-in features are enough to put my ideas into applications. Also the code structure is very neat, which I like very much.


Setup Postgresql, Python2.7 and Django using virtualenv on CentOS 6.3

Setup server is always a pain in the ass (if you don’t know chef or other similar tools). It took me two days to get my django app running on the VPS. My problem is there are a lot of tutorials on the website, but few of them cover everything. Also the tutorials are not always working for me, there are many weird problems that are very difficult to Google. So I made some notes for myself, hope it can help you as well.

Postgresql 9.2 on CentOS 6.3

  • Order of entries in pg_hba.conf is important. Always add the new user to the top of the list (that means before host all all).
  • If localhost gives you ident error, try connect using -h 127.0.0.1. Might be your VPS is using ipv6 address for localhost.
  • initdb with UTF8 support when your database is still empty.

Deploy Django using Apache + WSGI + Virtualenv

It seems simple from beginning because I have configured it with non-daemon mode before. This time I try to make it run in daemon mode. The tricky thing is there are many reasons for you to end up with a “ImportError: No module named django.core.wsgi” error. Here is a list you can go through to figure it out:

  • did you install mod_wsgi.so using yum? The module in yum is only for Python 2.6 so you have to compile your own mod_wsgi.so file.
  • run “ldd mod_wsgi.so” to check if libpython.2.7.so.1 shows up. if not, you have to recompile python2.7 and mod_wsgi with –enable-shared option.
  • if it still gets that error, check the permission setting for your virtualenv folder, is the user running daemon has enough privilege?

Template tag “follow_all_url” failed to render unfollow links in django-activity-stream

I have been trying django-activity-stream for weeks now and it is a great Django app. Everything worked great until today, when the tag “follow_all_url” stops rendering unfollow urls.

I tracked back the problem back to a function in class DisplayActivityFollowUrl in activity_tags.py file:

def render(self, context):
    actor_instance = self.actor.resolve(context)
    content_type = ContentType.objects.get_for_model(actor_instance).pk
    if Follow.objects.is_following(context.get('user'), actor_instance):
        return reverse('actstream_unfollow', kwargs={
            'content_type_id': content_type, 'object_id': actor_instance.pk})
    if self.actor_only:
        return reverse('actstream_follow', kwargs={
            'content_type_id': content_type, 'object_id': actor_instance.pk})
    return reverse('actstream_follow_all', kwargs={
        'content_type_id': content_type, 'object_id': actor_instance.pk})

context.get(‘user’) is supposed to get the request.user, but instead it gets nothing. A similar question on stackflow indicates that a setting might be missing, which is not the case but I think it is something wrong with how I created the view. I use class-based generic view and here is how my get_context_data() looks like:

def get_context_data(self, **kwargs):
	ctx = kwargs
	ctx['users'] = User.objects.all()
	return super(StreamView, self).get_context_data(**ctx)

Doesn’t look like it is the problem:(

I remember at first I used activity-stream with pinax-project-account project, and it worked with pinax-user-account app which also does something with the context_processing setting. In this new project I remove the user-account app and switched it with django-social-auth, maybe it missed something in my context?

I haven’t found a solution yet, but I will post the update to the problem here. For now, I think I can temporarily change that line to context.get(‘request’).user instead.


Update:
It was quite stupid actually, I have an object list in the template called “users”, so in a for loop each iterated item is called “user”. Change that to “people” solved the problem.


Bad UX from Prismatic

I was just like many others who signed up on Prismatic last week after read this post on TechCrunch. While I enjoyed some of the articles from it (I guess AI is still learning about my interests), there is one thing disturbs me when I checked my twitter today.

All of the articles I checked on Prismatic app were shared on my Twitter (and probably also could be on Facebook, but I didn’t grant them:)), instead of what I wanted to share. Unfortunately later I figured out how to turn off sharing on Prismatic’s website, but it is still very annoying. My friends must think my account was hijacked and turned into a spamming machine…

P.S. I will remove all those tweets later so you probably will not see them now.


Better Logging Support in TestComplete

I have been using TestComplete since May. The test suite has been running since June. Not sure if I used TC in the right way. Now it just acts as a giant Java interpreter to execute all the code in JScript  (a fairly slow one in my opinion). I haven’t used too many of TC’s functions.

Anyway if you have to use script test case, you need to deal with logging. TestComplete offers four different levels: event, info, warning, error. Here is my code in my common_function.js file, which stores most of the generic functions:

function log_msg()
{
var msg = join_log_args(arguments);
Indicator.PushText("[msg]: " + msg);
Log.Message(msg);
}

function log_event()
{
var msg = join_log_args(arguments);
Indicator.PushText("[event]: " + msg);
Log.Event(msg);
}

function log_warn()
{
var msg = join_log_args(arguments);
Indicator.PushText("[warn]: " + msg);
Log.Warning(msg);
}

function log_error()
{
var msg = join_log_args(arguments);
Indicator.PushText("[error]: " + msg);
Log.Error(msg);
}

function log_debug()
{
  if(vg_debug) //if debug is on
  {  
    var msg = join_log_args(arguments); 
    var cl_priority = 200; //low priority
    Indicator.PushText("[debug]: " + msg);
    Log.Message(msg, "", cl_priority);
  }
}

function join_log_args(args)
{
  var log_string = "";
  for (var i=0; i < args.length; i++)
  {
    log_string = log_string + args[i] + " ";
  }
  return log_string;
}

So if you want to log something, you just need to use:

log_error(log_prefix, "this is an error, error code:", error_code);

and it will automatically join all the input parameters together.


%d bloggers like this: