Are we able to create a system (AI) equally as complex as the system (our brains) we are using to create it?

These questions sound very deep and would maybe strike some people as being legitimate question, but they’re not.  When it comes to creating a system which exhibits generalized intelligence, we have the distinct advantage of having a fully functional system which does exactly what we’re looking to do… our brains.  Being able to disect brains, and watch (to some extent) the brain in action means that we we are not simply given the task of “thinking about how we think”, we also get MANY biological clues as to how the brain performs the amazing tasks that it does.  This basically presents us with a puzzle of which we have a lot of the pieces and we have a decent idea of what the end picture should look like, but we have no idea how they go together or what pieces we’re missing.  I’m in no way trying to trivialize the task of creating artificial intelligence by comparing it to a puzzle, I’m saying that it’s not about “can we do it” it’s about “how and when will we do it”.

The most significant road-block to this problem being solved is not the problem itself, but the people (like me) who are trying to solve it.  Those who take interest in this problem are primarily software developers who see that ENORMOUSLY complex things can be done using computer software.  The problem is that software developers always want to “create a program that does X” and when you create artificial intelligence, you need to “create a program that does anything”.  Understanding the difference between creating a program which solves a problem and creating a program which solves all problems (or almost all) is where most AI projects fail.  You see the projects all the time which I call “supermaket navigation systems” which are written to do one thing and one thing only.  I’m sure that those in charge of the project originally meant to create the universal intelligence, but over time the developer’s need to solve the problem will take over and he/she will begin to focus on solving one problem.  This is the point at which the only thing the software can do is pick out objects from a pictures or navigate a supermarket.  These are great accomplishments, but they do not bring us any closer to truely generalized artificial intelligence.  You show me the software that can navigate a supermarket AND pick out objects from a picture (using the same system) and that is what I would consider true progress.

Just to be clear, I’m not saying that software developers can’t solve the problem, I’m saying that software developers need to know that they are their own worst enemy when it comes to solving this problem.  I firmly believe that it will be a software developer who figures out the secret and I also believe that it won’t be nearly as complex as most people believe.

The concept behind these plans is great.  You pay a little extra, your product breaks, it gets fixed or replaced.  Wow… how pleasant.  The truth of the matter, which you can clearly see in the Best Buy complaint post, is that when you go to actually use your extended warranty, they will try to make the process as hard as possible for you.  In my own experiences with Future Shop, I had a DVD player with the product service plan, however when that DVD player broke it took them two weeks to repair it for me.  Not only was I out of my DVD player for two weeks, when they gave it back to me, it still had the SAME PROBLEM! They didn’t fix anything at all! So I brought it back again and said, please fix the problem.  They again had the DVD player for two weeks. When I got it back, it worked to play one movie and then started back up with the same problems again.  At this point, I needed a DVD player (we were having friends over), so I just went out and bought a new one.  My time is much more valuable than a DVD player and therefore Future Shop got exactly what they wanted, I gave up.  The part they seem to be missing however, is that I will never buy their product service plan again, and I will tell everyone I know the experience I had with their product service plan.

I’m not sure why Best Buy and Future Shop seem to forget that they are losing customers daily by allowing this kind of poor customer service to continue.

Anyways… it’s cool.

Now, NetFlix is a pretty big company and I’m sure they have some pretty smart people working for them.  The question for people entering this contest is, just how smart are the Netflix engineers?

From a business point of view, this move by NetFlix is extremely smart.  It will make thousands of people try to solve an extremely difficult problem.  Since the problem has been worked on for years now and little progress has been made recently, it is more than likely that the next major advancement will come by moving away from the current top solution and using a completely different technique.  This contest should fuel new thought and innovation and I believe NetFlix should be commended for funding research in this way.

Now the question is… should I try?  A seemingly impossible task with a million dollar reward… that’s right up my alley.

I’ll have to look into it a little more…

In order for an operating system to be used by “the public”, it has to be relatively simple.  If they want something, they should have to click, answer a few questions and move on.  This is what Windows provides and this is what will be required of Linux before it is able to start taking significant market share.

Now, this won’t be a long ranting post, but what I do want to say is that Linux is almost ready.  I recently setup a drive array (software RAID 5) in Linux and found that with the most recent version of the Kernel, it was able to detect all of my hardware and required very little “hacking” to get working.  In my opinion hardware detection was the previous downfall of Linux, however from my recent experience, this problem is well on its way to being fixed.  More vendors are releasing Linux drivers and more legacy drivers for older hardware are being released all the time.  As the userbase grows, so will the support.

I now have a spare computer which I can use as my Linux desktop test.  I’m going to see how much of my day to day computer work I could do using a Linux desktop and what problems I run into.  I’ll be recording my experiences here.

Also, I’m heading out on vaction tomorrow, I’ll be going to California and will be stopping by Legoland.  I’ve never been before, so I’m hoping they will have some additions for my new Mindstorms NXT set.  I’m going to need more supplies if I’m going to make a better device than the chaingun.

The underlying problem here is that the big software companies (you know who) have tried to make the computer a friendly device which acts more like a co-worker than a typewriter.  People treat the computer like another person because it asks them questions and cute little paper-clips talk to them.  The function of asking questions and talking to them is simply a method of extracting information from people who can’t tell the computer what they want.  But from the user’s point of view, the computer is talking to them and therefore is doing things for them.  This is where things break down.

You see, so the user ends up thinking the computer is doing something for them when really they are using the computer to do something for themselves.  So when something goes wrong, it’s the computer’s fault, because the computer was doing it.

Now there are some cases where software or hardware is defective and in those cases, yes, the computer is to blame.  But when someone is using a program that is known to crash often, and they still continue to use it… can they really blame it when they lose their file?

1. Keep up to date
The best tip I can give you to have better results is to use the latest version of SPAMassassin.  You don’t need to have bleeding edge, but just make sure you aren’t using a version which is a year or two old.  Update your installation every few months if you can.  The newer versions of SPAMassassin are always adding new detection techniques and rules which help cut out a vast majority of SPAM.  I’ve found that newer features such as the URIBL are able to cut your false negatives down by 80%.

2. Use a third party ruleset.
The SPAMers find ways around the base rules in SPAMassassin all the time, so the more rules you can add, the better (as long as they are good rules).  SARE is a great resource for a constantly updated set of third party rules.  These rules are all tested and help to greatly increase the accuracy of your filter.  A script called RulesDuJour is also available which automatically downloads these rules for you and updates your installation with the latest ruleset at a scheduled time every day.  Having a constantly changing ruleset makes it difficult for the SPAMers to get around the filter.

3. Train your Bayes engine
SPAMassassin comes with a bayesian learning engine which allows it to use statistical methods to categorize a message a SPAM or HAM (not SPAM).  This engine is extremely accurate as long as it has the proper training.  In order to train the bayes engine you need to provide it with messages that have been reviewed by a human and are confirmed to be either SPAM or HAM.  You can manually train the engine using the sa-learn command, however it is far more efficient to use a front end to SPAMassassin like Maia Mailguard.  With Maia, it keeps a copy of all messages that pass through it so that if a message either gets filtered when it shouldn’t have or doesn’t get filtered when it should have, you can go back and mark the message as either SPAM or HAM.  This will then train the Bayes engine and it will then filter similar messages in the future.  I’ve really just touched on the Bayes engine in SPAMassassin and therefore you should look it up online and read more about it to find out what the best method would be for your environment.

4. Use the SPAM network detection methods
There are several blacklists and hash detection methods that can be used in SPAMassassin.  The two that I highly recommend are DCC and Razor.  These two products will help to increase the accuracy of your SPAM filter again by a large percentage.  Support for the products is builtin to SPAMassassin by default but they do require a client to be installed in order for them to start working.  A quick search online will provide many tutorials for getting them going.

Those are the big four tips I have.  There is a lot of other tuning you can do, but by making sure you are doing the above four things, your accuracy should easily be above 98%.  Using this configuration, I am currently seeing about 99.4% accuracy for detecting SPAM and only about 0.01% false positives.  If you have any other good tips for SPAMassassin, please leave a comment.

“Remember two months ago when you were fixing my printer? Well, now my Internet Explorer doesn’t work.”

This is a common example of a user with “related blame” syndrome.  What the user doesn’t quite understand is that they are creating a correlation based on the fact that both of these things are in the “technology” category and that there was a problem with each of them.  If we look at example which is not in the technology realm, it helps to demonstrate the depth of this issue.

“Remember how last week at lunch your soup was cold? Well now my fridge doesn’t work.”

This statement is almost exactly equivalent.  Soup is related to food, fridge is related to food, there was a problem with both of them, so the cold soup must be related to the broken fridge.

Now, I understand this is because technology is difficult to understand for some and that they don’t understand the underlying systems involved, but this does not explain why they need to create any correlation at all.  If you don’t know anything about the subject at hand, why would you attempt to make a correlation? If I get the flu the day after I had a salad containing cucumber, I don’t draw a correlation between cucumbers and the flu.  I don’t know anything about the flu or how a cucumber might be related to it and therefore making that correlation doesn’t even enter my mind.

This problem affects the mental well-being of computer professionals around the world and therefore must be stopped.  Please help fight related blame syndrome in the technology industry by allowing those afflicted with this problem to read this article and understand exactly what they are saying.

Many people in the tech industry are fans of one company or another, but I’m a fan of one product over another.  For example, I’m a big fan of Google’s recent move into the office productivity arena because it gives Microsoft, who is currently dominating that market, some healthy competition.  This forces Microsoft to make a better Office suite and maybe even lower their prices.

Unlike a lot of tech people however, I’m not all about sticking it to Microsoft.  I like the fact that Microsoft is trying to improve their search engine to compete with Google.  This forces Google to step up to the plate and deliver a top quality search engine.  I also like the fact that Microsoft is releasing a portable media player named Zune which forces Apple to make sure the iPod is always moving forward and improving.

So when people ask me why I like Linux so much, I tell them that it’s because Linux is one of the only operating systems that has a serious shot and competing with Windows…. and competition is a good thing.

5. Doing day-to-day work as an administrator
Logging into your system as an administrator/root just to check your email or surf the web is a very bad thing.  You should only be using administrative privileges when you need them.  Using commands like su in *nix or runas in Windows is the best way to get admin privileges only when you need them.  If you’re logging in as an administrator, an attacker simply has to send you a malicious email or lure you to a bad website to gain complete control over your system and maybe even your network.

4. Vulnerable email client
If your e-mail client is not fully patched with the latest vendor security patches, you’re asking for a world of trouble.  When someone is able to simply send you an email and have it auto-execute whatever code they want, this is a bad thing!  Patch your email client often and if you’re using a common email client like Microsoft Outlook, it’s even more important.

3. FTP server with simple account passwords.
In one of the companies I used to work for, we ran a small webhosting server farm.  The logs on these servers would show invalid FTP login attempts.  At least once a week (if not more), we would see an attacker probe usernames/passwords.  They would try a few hundred common username/password combinations to attempt to get in.  The result when they do get in is usually several hidden directories in which they put movies/music/pirated software.  Make sure your FTP server doesn’t have any “default” accounts and all of your common accounts have strong passwords.

2. Vulnerable web browser.
Using an unpatched web browser is like leaving the keys to your parked car on the hood.  You won’t definitely have your car stolen, but the odds are good.  The internet has become a breading ground for spyware and viruses.  A large majority of spyware infects people’s system using web browser vulnerabilities.  Make sure to always download and install recent Firefox updates, visit Windows Update to patch Internet Explorer or download all security updates for whatever browser you do use.

1. Blank local administrator or root password.
The number one easiest way to get hack is to have a blank local administrator or root password set on your system.  You may think that it is quite obvious, but it can be easily overlooked.  I’ve seen systems hacked in less than an day when the Citrix server went online with a blank local administrator password.  This kind of thing was easily missed because the system was only logged into using domain accounts and the manufacturer installation CD had set the administrator password to blank.  Always double check all accounts that have access to your system for reasonable passwords.  No system in your network should have a blank administrative password!

The best new feature is definitely the “Ultra Sonic” sensor.  It allows the robot to judge depth and see motion.  It’s way better when your robot can see how far it is from the wall rather than having to bump into it before it knows it is there.

If you enjoy spending many many hours trying to accomplish a small task (like me), then you’ll enjoy this.  Even though you are somewhat limited with the sensors you have, you can still do some very cool stuff.

So when I got my set, I built the basic robot.  This took a few hours.  The basic robot allows you to put a ball in front of it and it drives straight at it and then picks it up with its claws when it gets to the ball.  That was neat for about 10 minutes.

I then decided I needed to try and do something myself so I thought, how hard could it be to modify the robot to make it locate the ball first and then go and pick it up.  Well… about 6 hours later… I finally got it to do this.  Going through an excercise like this shows the complexity of robots and gives you a lot of respect for the guys who make robots that actually do useful things.

Anyways… here is what my robot ended up looking like.  The design was only slightly modified from the one in the instructions, but the program I wrote was completely different.

More robots to come…

The problem is that these people honestly believe they are computer experts. Why? I’m not too sure.  At what point do these people or their relatives award them the “computer expert” designation?

I just wonder what makes computers so different that experts are so easy to come by?  It doesn’t seem to apply to other professions, so why does it apply to this one?

I guess in the end, if you can’t beat ‘em, join ‘em. So I here by declare myself the following:

• Plumber - I can use a plunger
• Doctor - I’ve applied one or more band-aids
• Lawyer - One time I got in an argument
• Stunt driver - I hit a curb once
• Accountant - I pay my bills
• Clown - Sometimes I dress funny

I hope that everyone from all of those professions accepts me into their field with open arms the same way I have accepted all of the “computer experts” into my field.  Hey wait, now that I think of it, I think I fit into one more category… Gynecologist!

One day when this blog becomes so very infinitely popular, kids will most likely have to memorize this post and recite it at the start of every class.

You’ll see… you’ll all see!!