Archive for category Programming

You know its finals time…

Analytics Pageview Rankings

A few quarters ago, I took PHIL 230. Our final exam had a quote id section so I quickly wrote a quiz app which I distributed to the class. Since then, the professor has been circulating the link as a study tool. So once a quarter, my web traffic spikes a day before each final exam :).

Last Quarter's Statistics

This Quarter's Statistics

Pageview spike this quarter

Last Quarter's Statistics

If you want to try it out, swing over to: http://www.prenticew.com/phil230quiz/.

DISCLAIMER: I wrote this in all of two hours or so and never looked back… I know there are some backend flaws, so don’t break it please :)

No Comments

Erasing Iterators from STL Containers (STL Vector, etc.) in a Loop

for( vector<aType>::iterator it = aVec.begin(); 
      it != aVec.end(); 
      it = (*it).shouldDelete()?aVec.erase(it):it+1){
   // do stuff
}

, , , , , , , , , ,

2 Comments

eWeek 2010!

I have not posted for a while, mostly because this quarter has been busy as hell! Most of it thanks to eWeek (Engineers Week). I’m on of the directors for eWeek and that means I’m responsible for overseeing and planning over 22 events over 4 days.

We did some pretty cool stuff for eWeek, the biggest thing in my opinion was our awesome barcode system! In the past, we used “passports” as a marketing tool to encourage people to attend our events. At the beginning of the week, people can pick up a passport and at every subsequent event they attend they get a stamp. The more stamps they collect, the more prizes they become eligible for (including a HDTV!). While this system was effective in increasing event attendance, I saw an opportunity to collect more useful statistics by using a barcode tracking system.

I developed a web application in collaboration with Brian Oppenheim which allowed us to track who came to our events and when they came to our events. This data allowed us to analyze the effectiveness of our publicity, popularity of events, the most effective times to schedule events, major (EE, ME, CSC, etc.) representation at events, etc. etc.

I’ll probably have to write a blog post about this system eventually… but for now… enjoy these photos!

(also a video to come).

eWeek 2010 Website: http://esc.calpoly.edu/eweek

Photos on Flickr: http://www.flickr.com/photos/echo0101/collections/72157623478386422/

Day 1 (Tuesday):

Day 2 (Wednesday):

Day 3 (Thursday):

Day 4 (Friday):

, , , , , ,

No Comments

Fractal Landscapes

CPE 471 Fractal Landscape

This might be the most interesting graphics project I’ve worked on all quarter. Most of the other projects were pretty boring but this one… I just can’t seem to put down. I finished it at 2AM last night, but managed to stay up till 6AM tweaking it for performance. I didn’t even realize the time pass. Anyway, I spent most of today further modifying the lighting and various characteristics to make the scene look more realistic.

So first, if you haven’t heard of fractal landscapes, here is a little background reading: http://en.wikipedia.org/wiki/Fractal_landscape

The initial rendering: Recursion depth 9.

CPE 471 Fractal Landscapes

CPE 471 Fractal Landscapes

The first aesthetic addition I added was fog. Fog is very trivial to add in OpenGL. Then I added sky. Sky was pretty simple too, it is a square with a gradient sitting in the background.

CPE 471 Fractal Landscapes

CPE 471 Fractal Landscapes

CPE 471 Fractal Landscapes

At this point, I was pretty satisfied… but I was unhappy with the resolution of the landscape… the polygons were still too large and you can see where they join. So, in order to increase the recursion depth, I had to optimize my program further. The first optimization I had to do was to speed up the rendering. For 9 levels of recursion, about 262,144 polygons are drawn per frame. For 10 levels of recursion, about 1 million polygons are drawn per frame. Initially, without optimization to my original approach, I was able to get the program to run at an acceptable frame rate at 8 levels of recursion. After simple optimization, I got to 9. And after working on it today, my program handles 11 levels of recursion (just not smoothly :P). I think 11 looks pretty good so I actually have not tried more. The trick was to reduce the number of calculations that needed to be done at the time of rendering. I essentially performed all the calculations for the vertices, normals and colors before hand and stored them in a linear array. Then during rendering, I simply load them all into a GL_QUAD_STRIP.

After I was satisfied with this, I also played around with lighting. Initially, the scene was lit by a single light source from behind the camera. This is not very realistic because the sun isn’t always directly behind you. Anyhow, I turned that light off and added a light source from the right as well as a orange-ish light from the back to give the mountains some backlight and define the shape a little.

Here are some examples rendered at recursion depth 11.

CPE 471 Fractal Landscapes

CPE 471 Fractal Landscape

It was interesting to think about to light this landscape. Photographers constantly think about how their subjects are lit and how to light them, but rarely do they ever have the opportunity to light an entire mountain!

3 Comments

Using Variadic Functions in C

I don’t really need to write on how to use va_list, va_start, va_end, va_arg because other tutorials or references do a good job of explaining it already… however, here are some notes for wrapping Varadic functions.

Firstly, note the difference between:

void myFc( int arg1, ... )

and the va_list version:

void vmyFc( int arg1, va_list args)

gnu stdc libraries (printf, etc) wrap the va_list versions (vprintf, etc) with Variadic versions (printf, etc) with the following pattern:

int vfunc(int arg1, va_list vargs){
   // do real work
}
int func(int arg1, ...){
   int retval;
   va_list vargs;
   va_start(vargs, arg1);
   retval = vfunc(arg1, vargs); 
   va_end(vargs);
   return retval;
}

When wrapping va_list functions, it is important to consider that va_list is consumed and so in the case where you will be using your va_list for multiple functions, you’ll need to save the original pointer.

GNU C doc for stdarg.h — Note the __va_copy macro.

For example, wrapping the snprintf function:

//to find the length of the string, you pass null and length 0 to the function:
len = vsnprintf(NULL, 0, fmt, vargs);

//do the allocation
str = (char*) malloc(len+1);

//and finally read the string:
vsnprintf(str, len+1, fmt, vargs);

note that we use the va_list version of snprintf (vsnprintf).

the 2nd snprintf may (depending on stdarg implementation) cause a segmentation fault. The correct/safe way of doing it would be to copy vargs and use the copy in each snprintf operation:

#ifdef __va_copy
   __va_copy(save,vargs);
#else
   save = vargs;
#endif
len = vsnpritnf(NULL, 0, fmt, save);

str = (char*) malloc(len+1);

#ifdef __va_copy
   __va_copy(save,vargs);
#else
   save = vargs;
#endif
vsnpritnf(str, len+1, fmt, save);

But as my awesome co-worker Geoff asserts, its always better to keep it simple and perhaps there’s a way to accomplish what you’re trying to do without Variadic functions. Check out my other post.

PST~ this was just a brain dump of what was on my mind as I was coding today… if you find this useful and/or find some info lacking OR just incorrect, leave a comment so I can fix it.

, , , , , , , , , , , , , , , , , , , , ,

1 Comment

Fun with Variadic functions

If you’re looking for info on implementing Variadic functions, its not here, but you can find it here (in C) or on wikipedia.

Today @ work, I wrote this awesome library to wrap fprintf, printf, etc and check that line lengths do not exceed 80 chars per line (if length exceeds 80, output warning message to stderr). The first problem I ran into was how to handle the variable number of arguments that can trail in a printf function. stdarg.h comes to the rescue with Variadic functions! Then I ran in to some problems with using the va_list more than once (since the arguments get “consumed” as the va_list pointer gets incremented as each argument is accessed). So this is resolved by copying the va_list pointer using a gnu extension, but my code has to be compiler independent, so an ifdef gets added.. blah blah blah. Because of how complex this solution is getting… I consider using snprintf and then printf the output of that (so i can get the length of the string from snprintf)… but this means I’d have to go through this 3-step process each time something is output (which is bad both because its a pain in the ass and for maintainability).. and so I’d want to wrap this with a macro. But with the variable number of arguments… there wouldn’t be an easy way of doing that either. I finally get my original idea with the Variadic function working… and then I realize, that each output generated by printf is not always one line, I can printf multiple lines… or use multiple printfs to generate one line. AHH.. so, introduce a static int that counts chars per line… OR… all this is being dumped to a file anyway… the simplest solution… JUST READ THE FILE at the very end and check line length!

The lesson here, keep it simple 😛 Though my Varidic solution is far more elegant IMO… though excessive for what I was trying to accomplish.

, , ,

1 Comment

Stripping C/C++ Comments

Here’s some code to strip comments from a c/c++ file. Code is adapted from a posting at http://stackoverflow.com/questions/241327/python-snippet-to-remove-c-and-c-comments

import re

# adapted from: http://stackoverflow.com/questions/241327/python-snippet-to-remove-c-and-c-comments
# strips c/c++ comments

def strip_comment(text):
    rep = r'//.*?$|/\*.*?\*/|\'(?:\\.|[^\\\'])*\'|"(?:\\.|[^\\"])*"'
    pattern = re.compile(rep, re.DOTALL | re.MULTILINE)
    return re.sub(pattern,
        lambda match:(match.group(0),"")[match.group(0).startswith('/')],
        text)

, , , , ,

1 Comment

Restoring files or folders in SVN

To recover a folder or a file in svn, you can use the copy command to copy the folder from the revision of the repository before the file or folder was deleted (or modified). The syntax can be a bit confusing but here’s how it can be done. The first flag -r (i’m not sure if you need this the way this command is formatted) specified the revision you copy from. The next argument is the source and it needs to be the full path to the file or folder you’re recovering followed by @ and the revision number. then the location the to copy the data to (which can be just . for this folder).

Example:

svn cp -r 111 https://server/svn/path/to/folder@111 .
svn ci -m "restored file or folder"

, , ,

1 Comment

Named groups in Python regular expressions

Regular expressions are very powerful. Named groups in Python re make your re’s and code very readable and powerful. Here is a simple example but imagine how useful this would be for parsing a dataset such as a cvs list or datafile.

import re

def main():
   str=  "james:12;google."
   match = re.match(r"(?P<name>[^:]*):(?P<age>[^;]*);(?P<company>[^.]*)."
         ,str)
   print "name", match.group("name")
   print "age", match.group("age")
   print "company", match.group("company")

if __name__ == "__main__":
   main()

, ,

No Comments

Python Talk

I gave a talk on Python for cplug yesterday.  

Check out the slides here.

CPLUG: http://www.cplug.org/
SLIDES: http://prenticew.com/talks/pytalk09

During the talk, I wrote a simple Python script to pull content off the web and parse the data… here’s the jist of what I did:

First, lets pull a webpage off the internet:

import httplib
def get_webpage():
   conn = httplib.HTTPConnection('en.wikipedia.org')
   conn.request("GET","/wiki/Python_(programming_language)")
   rd = conn.getresponse()
   print rd.status, rd.reason
   return rd.read()

This function creates a HTTPConnection object for en.wikipedia.org and the connection object is stored in conn.
We then do a GET request for the Python wiki page. The result of the request is stored in the connection and we can access the status by calling getresponse() which returns a HTTPResponse object.
The status can be accessed with .status and .reason and the data can be accessed with .read().

This yields the plain-text html of the wiki page. This is not very interesting or useful so lets do something else with this data… lets write a frequency counter:

 def get_freqct(data):
     wordlist = data.split(' ')
     freqct = {}
     for s in wordlist:
       if s not in freqct:
         freqct[s]=1
       else:
          freqct[s]+=1
     return freqct 

We can pass the data (a string) we got from the first function to our get_freqct function. The function first uses the built-in string function to split the string by a white-space delimiter returning a list of words. We then iterate through the wordlist and generate the frequency count using the dictionary data type. At this point we have something fairly interesting but simply printing out this list is fairly cluttered… lets sort it!

You can quickly sort the contents of this dictionary with the sorted function:

import httplib
from operator import itemgetter

sol = sorted(d.items(), key=itemgetter(1))

This statement takes the items in d (the dictionary) and returns a list of tuples (key,data) and is sorted by the data field of the tuple using the itemgetter function. So you’ll end up with a sorted list of tuples ordered by the data field.

Then we can print the list with the following for loop:

   for word,count in sol:
      print word, ":", count

This for loop unpacks the contents of each of the tuples in the sorted list (sol) into the variables word and count. The variables are then printed with the print statement.

If you run this code… you’ll realize that a lot of HTML tags (or parts of HTML tags) get counted. This is not very desirable so lets filter them out using a regular expression!

data = re.sub(r'<[^>]+>','',data)

This regular expression takes the raw data (string) returned by the get_webpage function and replaces each occurrence of an HTML tag with an empty string.

Deconstructing the regular expression:
<- matches the ‘<‘ symbol
[^>]+ – matches one or more of anything except the ‘>’ symbol (where + means one or more)
>- matches the ‘>’ symbol

…and put it all together:

#!/usr/bin/python
import httplib
import httplibfrom operator import itemgetter
import re

def get_webpage(site,page):
   conn = httplib.HTTPConnection(site)
   conn.request("GET", page)
   rd = conn.getresponse()
   print rd.status, rd.reason
   return rd.read()

def get_freqct(list):
    freqct = {}
    for s in list:
      if s not in freqct:
        freqct[s]=1
      else:
         freqct[s]+=1
    return freqct

def main():
   data = get_webpage('en.wikipedia.org',"/wiki/Python_(programming_language)")
   data = re.sub(r'<[^>]+>','',data)
   d = get_freqct(data.split(' '))
   sol = sorted(d.items(), key=itemgetter(1))
   for word,count in sol:
      print word, ":", count

if __name__ == "__main__":
   main()

The following is a snippet of what the script would yield:

language : 24
code : 24
which : 24
by : 27
Retrieved : 32
with : 32
are : 33
as : 38
on : 50
for : 51
in : 64
is : 80
to : 92
a : 98
Python : 103
and : 122
of : 125
the : 144

, , , , , , , , ,

No Comments