Wednesday, November 11, 2009

Face Detection "Hello World" in IronPython using Emgu CV and OpenCV

While following the instructions for setting up IronPython to use Emgu CV:
http://www.emgu.com/wiki/index.php/Setting_up_Emgu_CV_and_IronPython
I had a few problems. I started by trying to use latest versions (2.0) of these packages but kept getting dll load errors like:
"SystemError: The type initializer for 'Emgu.CV.CvInvoke' threw an exception."
I put my best google-foo on it and tried things like upgrading .Net to 3.5 but no dice. So I backtracked to Emgu CV 1.5 and OpenCV 1.1 and was able to getting it working:


import sys
import clr

sys.path.append(r"C:\OpenCV11\bin")
sys.path.append(r"C:\apps\Emgu1.5")

clr.AddReferenceToFile("Emgu.CV.dll")
clr.AddReferenceToFile("Emgu.Util.dll")
clr.AddReferenceToFile("Emgu.CV.UI.dll")
clr.AddReference("System.Drawing")

from Emgu.CV import Image, HaarCascade
from Emgu.CV.UI import ImageViewer
from Emgu.CV.Structure import Gray
from System import Byte
from System.Drawing import Size
from Emgu.CV.CvEnum import HAAR_DETECTION_TYPE

detector = HaarCascade(r"C:\OpenCV11\data\haarcascades\haarcascade_frontalface_alt2.xml")

img = Image[Gray, Byte](r"C:\OpenCV11\samples\c\lena.jpg")

# http://www.cognotics.com/opencv/servo_2007_series/part_2/page_2.html
#http://www.emgu.com/wiki/files/2.0.0.0/html/11c784fc-7d30-a921-07ec-ecdb7d217bbe.htm
scaleIncreaseRate = 1.1
minNeighbors = 3
minSearchScale = Size(img.Width/8, img.Height/8) # As a ratio of the image size:
#Or a fixed pixel size: minSearchScale = Size(100,100)
iChannel = 0 # only 1 channel

objectsDetected = img.DetectHaarCascade(
detector,
scaleIncreaseRate,
minNeighbors,
HAAR_DETECTION_TYPE.DO_CANNY_PRUNING,
minSearchScale
)[iChannel]

for obj in objectsDetected:
print "Object Rectange: %s" % obj.rect
img.Draw(obj.rect, Gray(255), 1)

ImageViewer.Show(img)

A couple notes compared to the Emgu wiki version--I was able to get everything working without having to disturb the IronPython install (I didn't copy any files into the IronPython start directory). I simply linked used sys.path to add the locations where the dll's live. Also, I did explicit import for everything. It's a bit more work to figure out what needs to be imported, but it avoids namespace pollution issues. Also, the img.Draw line needed a few tweaks. I had to drop the [float] bit from the original. And with the original version, I was also getting an error on this same line, saying:

TypeError: expected MCvBox2D, got MCvAvgComp

I was able to work around this by passing in the ".rect" rectangle attribute to the Draw method (I'm guessing the api has evolved since the 1.4 Emgu version used for the original). Lastly, I was getting two objects detected--so I used parameters to the detector to only get the single face in the image (basically by telling it to use a larger minimum detection size, minSearchScale). Anyway, now it works the way I was hoping.

And here's some bits about the environment, and the results it prints out:

C:\temp>"C:\Program Files\IronPython 2.6\ipy"
IronPython 2.6 (2.6.10920.0) on .NET 2.0.50727.3603
Type "help", "copyright", "credits" or "license" for more information
>>> execfile(r"C:\work\EmguTest.py")
Object Rectange: {X=222,Y=206,Width=163,Height=163}

And lastly, I'd be remiss if I didn't include lovely Lena, with face detected (in B&W, sorry about that Lena).

UPDATE:
When I tried to port the above configuration to my target server, I started getting that Emgu.CV.CvInvoke exception again. I upgraded the server to .Net 3.5 but that didn't get it. Then I applied the Microsoft Visual C++ 2005 SP1 Redistributable Package (see link) to the server and then it ran without error. So to summarize, I was able to get this running using Emgu CV 1.5, OpenCv 1.1, and the Visual C++ 2005 SP1 package installation. It runs on IronPython 2.0.1 and 2.6. I'm not certain if a .Net 3.5 had any effect:

Microsoft Visual C++ 2005 SP1 Redistributable Package Download

Monday, August 17, 2009

Annoying cache problem in FF3.0.x

"If it ain't broke, fix it till it is"
Of course nobody would do that intentionally, but what if something looks broken. Well then you might start throwing the kitchen sink at it and actually break it--that's my recent experience doing some SharePoint branding. I'm making changes to the CSS to overwrite the background image for the brand. It works fine in IE but it keeps showing the image I'm trying to overwrite in FF (3.0.13 in this case). So I waste a couple hours between trying variations and replicating the problem with a simple test case--having no idea it was a caching issue. But when I couldn't replicate the problem, I used a completely different computer and realized it looked fine there on the same version of FF. Argh! All that time down the drain on something that wasn't broken. Mind you, I did suspect caching at one point early on and cleared out all the cache and tried restarting FF. Now onto the research phase... After spending some quality time with google on this subject, it seems FF3 did have some caching issues. You can read about it here:

http://www.west-wind.com/Weblog/posts/469125.aspx

And here:

http://forums.mozillazine.org/viewtopic.php?f=25&t=673135&sid=7859918dca9a1722635e233ef6f401db

The bottom line--I tried everything I could think to do with FF to get it to stop caching (about:config browser.cache settings, clearing cache umpteen different ways, physically deleting cache files, appending url parameters onto the files in question) but couldn't break the back of the beast. The one thing that finally solved it was creating a new profile for FF and using that (as suggested by jscher2000 in that second link). Here's how:

http://kb.mozillazine.org/Creating_a_new_Firefox_profile_on_Windows

As for the root cause, I'm not sure. The various posting on this show it's pretty complicated--possibly involving interactions between thinks like the type of request (GET, POST), the transport (http versus https), the cache header settings, if you have other tabs open, and installed addins (especially FireBug). The nice thing about the additional profile is I can keep it plugin free--which may be why it solved the problem. Hopefully this won't be a problem anymore in FF 3.5.

Interestingly, using about:cache and FireBug, I was able to determine that the CSS file that didn't seem to be having any effect was in fact being downloaded. But when I looked at the computed CSS in FireBug, it wasn't showing the change. That doesn't necessarily mean it's FireBug causing it, but it could be the Heisenberg Uncertainty Principal is at play.

UPDATE
A coworker suggested I try Ctrl-Shift-R which is supposed to completely bypass cache. So I fired up the handy Tamper Data FireFox plugin since it shows the "Load Flags" of each requested URL (presumably FireFox's decision about what action it needs to take regarding the request URL--pull from cache, download, etc). A quick test of this on a blogger.com page showed that it did indeed eliminate most cached elements with a status of LOAD_BYPASS_CACHE. However, a few URLs, in this case images of type .png and .gif, still had a LOAD_ONLY_IF_MODIFIED status without any further followup showing up in Tamper Data. So while Ctrl-Shift-R does resolve most caching, it is still doesn't eliminate it completely.

Friday, April 17, 2009

SharePoint Feature Scope Changes Ignored

The Problem

I ran into a situation where I had deployed a broadly scoped feature and then narrowed the feature scope down, but the feature was acting like it still had the original scope settings. The feature was a delegate control to deploy JQuery using the AdditionalPageHead delegate control, just like Jan Tielens described here. When I had the feature scoped to the site collection (Scope="Site"), it was auto-deploying across the site collection as expected. But then when I tightened the scope down to specific websites (Scope="Web") and added a Scope="false" to make it require manual activation, it was behaving as if the it was still scoped at the site collection and was still auto-deploying. This continued even when I completely deleted the solution.

The Fix

After completely deleting the solution, I updated the solution and feature with new GUIDs. Thereafter, it behaved as expected. It seemed to behave like some remenant of the solution wasn't fully removed by from the solution store even though the solution was deleted. Here's a related forum posting on the issue. If it was something I did wrong, then I doubt changing the feature and solution GUIDs would have fixed it--that's my story anyway.

I'll also mention that I tried developing this with VSeWSS 1.3 but was frustrated by it always creating an assembly (even for the "empty" project template) which couldn't be removed without breaking things. I fell back to STSDEV which is remains my tool of choice.

Friday, March 27, 2009

A Basic Example of JQuery's UI.Dialog Dynamically Loading an URL with an IFRAME

Given the momentum that JQuery has had lately with .Net, I wanted to try out loading a form into a "lightbox" as an alternative to writing a webpart for SharePoint. So a first step is just to get the "lightbox" approach working. I used the JQuery UI Build Your Download page to build a barebones download for version 1.7.1. This download gives a nice bundle with demo pages--however I didn't see any pages with an example of dynamically loading an URL using an IFRAME. Searching the web turned up lots of stuff, but between different versions of the toolkit, options for using other plugins, and different approaches--I didn't quite find a nice simple example. So here goes.

Upon unpacking the download archive, you'll get a folder structure. I put my sample html files along side the html files in \development-bundle\demos\dialog, so if you're following along, that's where you should put the samples below.

In this first sample, the IFRAME is placed inside a DIV. In the second sample, the IFRAME is dynamically generated into the DIV--so it doesn't have to appear in the DIV content intially. In either case, there is some boilerplate css and jquery files that are loaded (they are pretty small in size compared to some other javascript libraries fortunately). Both samples also show the $(document).ready() function used to specify settings for the dialog. Then a links onclick method invokes the showDialog method. When the IFRAME is statically part of the div tag, it's src attribute is simply set in shoeDialog using JQuery's attr method.
<!DOCTYPE html><html><head>

<link type="text/css" href="../../themes/base/ui.all.css" rel="stylesheet" />

<script type="text/javascript" src="../../jquery-1.3.2.js"></script>
<script type="text/javascript" src="../../ui/ui.core.js"></script>
<script type="text/javascript" src="../../ui/ui.dialog.js"></script>

<script type="text/javascript">
function showDialog(){
$("#divId").dialog("open");
$("#modalIframeId").attr("src","http://www.google.com");
return false;
}

$(document).ready(function() {
$("#divId").dialog({
autoOpen: false,
modal: true,
height: 500,
width: 500
});
});
</script>
</head>
<body >

<a href="" onclick="return showDialog()">Show the Dialog</a>

<div id="divId" title="Dialog Title">
<iframe id="modalIframeId" width="100%" height="100%"
marginWidth="0" marginHeight="0" frameBorder="0" scrolling="auto"
title="Dialog Title">Your browser does not suppr</iframe>
</div>

</body>
</html>


For the second sample, just a few lines are different--removing the IFRAME from the DIV, and instead creating it in the showDialog method.

<!DOCTYPE html><html><head>

<link type="text/css" href="../../themes/base/ui.all.css" rel="stylesheet" />

<script type="text/javascript" src="../../jquery-1.3.2.js"></script>
<script type="text/javascript" src="../../ui/ui.core.js"></script>
<script type="text/javascript" src="../../ui/ui.dialog.js"></script>

<script type="text/javascript">
function showDialog(){
$("#divId").html('<iframe id="modalIframeId" width="100%" height="100%"
marginWidth="0" marginHeight="0" frameBorder="0" scrolling="auto" />').dialog("open");
$("#modalIframeId").attr("src","http://www.google.com");
return false;
}

$(document).ready(function() {
$("#divId").dialog({
autoOpen: false,
modal: true,
height: 500,
width: 500
});
});
</script>
</head>
<body >

<a href="" onclick="return showDialog()">Show the Dialog</a>

<div id="divId" title="Dialog Title" />

</body>
</html>

In either case, you can specify attributes on the IFRAME to control standard IFRAME features, and you can also use different options for the ui.dialog, described here (the JQuery documentation is quite good). Currently, the samples only use scrolling when required.

Wednesday, March 11, 2009

No BackStage Pass for Nested Classes in C#

In java land, I used to use nested classes to simplify a class that was getting complicated. If a class grew a cluster of fields and logic related to a single concept, I would create an inner class to encapsulate all that thus making the outer class simpler to understand--basic OO modularity/cohesion principles. Yes, some folks don't like inner classes and in java the compiler actually opens up the access modifiers of the fields--but I found it useful. But in C#, the nested class doesn't share any special access into the outer/enclosing class (perhaps for security reasons?). The workaround seems to be to pass in an instance of the outer class. The problem with this is that without special access to the internals of the outer class (fields, etc), you have to start modifying the access modifiers of the outer class, or start pushing all the state into the nested class via constructors or methods/properties. But that defeats the purpose--the idea was to keep the API to the outer class the same and keep the change encapsulated between the outer and nested class. The workaround I'm using is to have a special method on the outer class that has all the business logic needed to construct the nested class. Not perfect, but it seems like a decent compromise.

Friday, February 20, 2009

Calling an Oracle Stored Proc from ASP.Net

Here is a basic example of calling an Oracle Stored Proc from ASP.Net using a ref cursor to return the result. Basically, I installed the Oracle Data Provider for .Net downloaded from Oracle (just google "oracle .net"). I used the 11g provider even though I'm on a 9 database (it works fine so far). After setting up a connection to my Oracle database, I dropped a List View onto a page and bound it to a table in the schema just to display some data. Asp.Net does it's magic behind the scenes to compile the necessary subclasses and wire up connections and what not for everything to work. But at this point, the code behind class is basically empty. It turns out you can dig out the connection property that was generated by Asp.Net. Here's a little helper method that reuses the "MyConnection" property to serve up fresh OracleConnections:
protected OClient.OracleConnection getOracleConnection() {
return new OClient.OracleConnection(MyConnection.ConnectionString);
}
So using this, here's the code to retrieve a value from a ref cursor filled in by a stored proc:
using System;
using System.Data;
using Oracle.DataAccess.Client;
using Oracle.DataAccess.Types;
using OClient = Oracle.DataAccess.Client; // (1)

public partial class Class1 : System.Web.UI.Page {

protected void Page_Load(object sender, EventArgs e) {
using (OClient.OracleConnection oConn = getOracleConnection()) { // (2)
using (OClient.OracleCommand oCmd = new OClient.OracleCommand()) {

oCmd.Connection = oConn;
oCmd.CommandText = "mypackage.sayhello"; // (3)
oCmd.CommandType = System.Data.CommandType.StoredProcedure; // (4)

// Use name binding (instead of positional). Makes pl/sql parameter
// names part of the public api (can't change them without changing
// calling code), but moreclear and less error prone than positional.
oCmd.BindByName = true; // (5)

OClient.OracleParameter inSessionToken = new OClient.OracleParameter(
"in_session_token", OClient.OracleDbType.Long, ParameterDirection.Input);

OClient.OracleParameter outCursor_text = new OClient.OracleParameter(
"out_text", OClient.OracleDbType.RefCursor, ParameterDirection.Output);

// Should be first parameter added (see below)
oCmd.Parameters.Add(outCursor_text); // (6)
oCmd.Parameters.Add(inSessionToken);

inSessionToken.Value = 462; // just to show data getting passed in

oConn.Open();

Label1.Text = String.Format("MYPACKAGE.SAYHELLO: {0} at {1}",
oCmd.ExecuteScalar(), System.DateTime.Now); // (7)
}
}
}
}

Annotations:
  1. 1) There are various class name collisions between the Oracle.DataAccess.Client package and the System.Data.OracleClient package, and you need to reference both. Here the "using" alias helps make the code more readable while disambiguaging to the correct classes. Otherwise Oracle.DataAccess.Client would be sprinked throughout the code. It's really just syntactic sugar, but it helps.

  2. 2) Here the other (more important) application of "using" is managing the database resources. Note the nested calls for the connection and the command. This just saves boilerplate resource management code. Note that "using" calls IDispose on the underlying object. In the case of ADO, this should either return the connection to the pool (if pooling) or close the connection if not-\-assuming the provider was correctly written.

  3. 3) This is where the package.procedure is specified. Of course, you may just be using a procedure directly without a package (just use the procedure name by itself).

  4. 4) This is required to let the driver know you are calling a stored proc

  5. 5) I'm using named parameters, otherwise you could use positional parameters like :1, :2 , etc.

  6. 6) As I explain in the comments, it's safest to just add the ref cursor parameter first for the ExecuteScalar to work properly.

  7. 7) Note, that it doesn't make a lot of sense to spin up a ref cursor just to get a single "cell" of data out of it--you'd probably just want to return a varchar2 from a function instead. However, it's a quick way to show that it works. When a single value is all that's needed, ExecuteScalar can be used.
Here's what the stored proc looks like:
PACKAGE BODY "MYPACKAGE" IS

PROCEDURE "SAYHELLO" (
in_session_token IN PLS_INTEGER,
out_text OUT sys_refcursor) IS

BEGIN

open out_text for select 'Oracle says "Hello" for session token ' ||
in_session_token from dual;

END "SAYHELLO";

END "MYPACKAGE";
Of course, you'd want a package declaration (not shown, but it basically is just declares the procedures signature).

You can also use the OracleRefCursor class in conjunction with the
OracleDataAdapter class to populate DataTables and DataSets. Also, note that
the ExecuteScalar method for a refcursor must be returned from the function or the must be the first out bind parameter.
Oracle article on using ref cursors with .Net

The ref cursor is then accessed as either an OracleDataReader object or a
DataSet, by use of the OracleDataAdapter class.
Oracle article on using the ways you can handle a ref cursor

If you only need read-only data, use the OracleDataReader to populate a DataSet. If you need offline or updateability, use OracleDataAdapater.

Wednesday, January 28, 2009

Canonical Python __str__ method

Here's a python class that shows some approaches for building up a meaningful __str__() to display the class's data attributes (like overriding toString() in java). The first way is to show a specific subset (or ordering) of attributes. The second way (currently commented out) just shows all the classes attributessorted by name. I also show how to make it work with a subclass--either leveraging the superclass __str__() method or doing it all from within the subclass. Of course, it should be easy to adjust the output to your own needs (padding, etc).

Nothing earth-shattering here, but it can save some work if it's a big class with lots of attributes. The example should work with modern CPython, including Python 3:

class X:
def __init__(self):
self.a = "apple"
self.b = "banana"
self.c = "cranberry"

def __str__(self):

# to show specific variables
showList = ["a", "b"]

# to show include all variables in sorted order
#showList = sorted(set(self.__dict__))

return ("X(%i):\n" % id(self)) + "\n".join([" %s: %s" % (key.rjust(8), self.__dict__[key]) for key in showList])

class Y(X):

def __init__(self):
X.__init__(self)
self.d = "date"
self.e = "elderberry"

def __str__(self):
showList = ["a","d","e"]

# if you want to use the superclass as-is and supplement from this subclass
# return X.__str__(self) + "\n Y:\n" + "\n".join([" %s: %s" % (key.rjust(8), self.__dict__[key]) for key in showList])

# if you want to do everything from this subclass
return ("Y(%i):\n" % id(self)) + "\n".join([" %s: %s" % (key.rjust(8), self.__dict__[key]) for key in showList])

print(Y())

This outputs:

Y(12172528):
a: apple
d: date
e: elderberry


Okay, but maybe this can just be a helper method of some kind. The class name info can be pulled of of the self.__class__ object, and then the function can be more generic:

def str_impl(anInstance, displayAttributeList):

classLine = "%s.%s(%i):\n" % (anInstance.__class__.__module__, anInstance.__class__.__name__, id(anInstance))
return (classLine + "\n".join([" %s: %s" % (key.rjust(8), anInstance.__dict__[key]) for key in displayAttributeList]))

class XX:

showList = ["a", "b"]

def __init__(self):
self.a = "apple"
self.b = "banana"
self.c = "cranberry"

def __str__(self):
return str_impl(self, self.__class__.showList)


class YY(XX):
showList = ["a","d","e"]

def __init__(self):
X.__init__(self)
self.d = "date"
self.e = "elderberry"

This outputs:

__main__.YY(12172784):
a: apple
d: date
e: elderberry



Note that __main__ is the module since the classes are defined in the "main" scope. The correct module/package would print if I had used one. Again, you wouldn't even need to bother with this for a small class--but for a class with a lot of attributes (which might be a sign it's time to break it up anyway ;-), it saves some tedium.

Friday, January 9, 2009

Connecting to an Oracle Database from ASP.Net/SharePoint

So you’re writing your “Hello Oracle Database” from ASP.Net/SharePoint, and you get this error: “System.Data.OracleClient Requires Oracle Client version 8.1.7 or greater”.

Perhaps you don’t really want to install the 475 Mb Oracle Client on your system just to talk to a database. Some quick googling suggests you can just install the Oracle Instant Client, and if you don’t care about internationalization, you can even go with the Instant Client “Basic Lite” which gets you down to about 18 Mb:

http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/winsoft.html [32 bit version]

Yeah, it’s an order of magnitude larger than the jdbc jar you’d use if you were connecting from Java, but at least it’s not a half Gig. But what if you don’t even want to install it? This blog entry shows the 4 files you’ll need to from the instant client:

http://www.thewayofcoding.com/2008/02/c-net-programming-tip-connecting-to-an-oracle-database/

In my case, I’m actually developing a SharePoint web part. Therefore, I’d like to make the set of driver files universally available for anything deployed on the system. So I drop those four files:
· oci.dll
· orannzsbb10.dll
· oraocci10.dll
· oraociicus10.dll

into the system driver directory (in this case C:\Windows\System32). And now I’m good to go.
Note: I’m not sure if it’s related, but I did not have to change any permissions as seems to happen when you actually install the client (at least in some cases)—as described here:

http://moustafa-arafa.blogspot.com/2007/04/systemdataoracleclient-requires-oracle.html

I suppose it’s time I update my About Me to say I do .Net now too :-)