Archive for February, 2005

Packet fragmentation over the internet

Somebody wrote to me telling me that the code I included in my book to send and receive files via TCP sometimes hangs.

  You can improve the handling of files within the TCP file reciever program in two ways.

Firstly, instead of
"if (thisRead==0) break;"
it should be
"if (thisRead<=0) break;"
Furthermore, it is possible for packets of less than 1024 bytes to be sent at a time (this does not
happen over a local network, but sometimes happens over the internet). This could lead to file
corruption where packet-fragments are padded with zero’s in the output file.
So the code could be further improved thus:

System.Collections.ArrayList al = new ArrayList();

byte[] RecvBytes = new byte[Byte.MaxValue];

Int32 bytes;

Int32 totalLength=0



bytes = networkStream.Read(RecvBytes, 0,RecvBytes.Length);

if (bytes<=0) break;

totalLength += bytes;




= al.GetRange(0,totalLength);



[] returnedData = (byte[])al.ToArray((new byte()).GetType());



and also define the function cropByteArray thus:


byte[] cropByteArray(byte[] inputArray, Int32 crop)
byte[] outputArray = new byte[crop];
for (int i=0;i<crop;i++)
return outputArray;

Categories: Uncategorized

Experimental samplings

Some Mp3’s I dredged up from a few years ago. Amazing what you can do with a handful of sample CDs and Goldwave!


Categories: Uncategorized set live! was just set live!. It’s got a built-in POP3 client, so it can check email from one of about 20 UK and Irish ISP’s. I’m working on the site, so anybody knows any other POP3 servers, please drop us a note on the blog.

Also I noticed Clare Dillon’s blog 

Thanks Clare!

Categories: Uncategorized

SQL server 2000 – Full text search

After hunting on the net, and in books for ages, I figured out how to set up a full text catalog using stored procedures

exec sp_fulltext_database @action=’enable’
exec sp_fulltext_catalog @ftcat = ‘TableName’,@action=’create’
exec sp_fulltext_table @tabname=’TableName’, @action=’create’,@ftcat=’TableName’,@KeyName=’PK_TableName’
EXEC sp_fulltext_column ‘TableName’,’SomeTextColumn’,’add’

EXEC sp_fulltext_table ‘TableName’,’activate’
exec sp_fulltext_table @tabname=’TableName’, @action=’start_full’

You need to have the MSSearch (support service) running on the SQL server, and this doesn’t work in MSDE.



Categories: Uncategorized

Pop3 Servers

BT Connect
BT Internet
Direct Connection

Categories: Uncategorized

Signature applet

Just an idea I was working on, an applet to collect signatures online….

import java.awt.*;
import java.applet.*;

public class SignatureApplet extends Applet
 Point startPt, endPt;
 public void init()
 public boolean mouseDown(Event evt, int x, int y)
  startPt = new Point(x,y);
  return true;

 public boolean mouseDrag(Event evt, int x, int y)
  Graphics g = getGraphics();
  endPt = new Point(x,y);
  g.drawLine(startPt.x, startPt.y, endPt.x, endPt.y);
  startPt = endPt;
  return true;

Categories: Uncategorized

Some musings on cognitive computing….

Electronic vs Biological memory.


Memory is a scarce resource, in both electronic and biological forms. In our lives we perceive much more information through our senses than we could ever hope to remember. It is like trying to store the World Wide Web on a single computer.


Our brains prioritize information, providing quick access to important information, slower access to less important information, and we simply forget what unimportant details, such as what you had to eat for lunch two months ago. In a digital sense, if we were to store the Internet on a single computer, we couldn’t fit it all there, but we would start with the most useful and informative websites, and leave out websites such as “What I think of my neighbours cat” by Joe Nobody in Alabama.


Furthermore, our brains will generalize information, in order to save neuron-space. For instance, when you think of a horse, you would have a general image in your head, complete with head, tail, four legs etc. One can imagine the horse with or without a saddle, or in a variety of colours, but these are optional attributes. This is in stark comparison to how a computer would store images of horses, computers would have an index of a Grey horse, brown horse, Shetland pony etc., where a human would only have one image, but it would be general and accurate enough so that it is possible to instantly recognise a horse when seen.


Computers compress data too, in one of two ways, lossless or lossy. Lossless compression uses statistical analysis to reduce data entropy. A little similar to seeing six horses in a field, and remembering the number “6” and associating it with a horse, rather than remembering the image of the six horses. Lossy compression is where less visible, or less audible information is removed from either an image or sound. Digitally, this is performed using a mathematical harmonic analysis (DCT), to reduce low-amplitude harmonics from audio or images. Humans do this too, – You may know what a horse looks like, but couldn’t tell whether they have dimples on their noses or not.


One of the more interesting parts of human memory, is how information is prioritized. It happens without conscious intervention, but the process can be controlled with conscious thought. Most English-speaking people can remember maybe 20,000 english words, and their meanings, But only a few hundred people’s faces and names. This is because symbolic data takes less space than images – quite analogous to the digital world, where a small image could be 100,000 bytes in size, and a word consumes only a few bytes.


Regardless of neuron-space consumption, all information is prioritized in the same way. You might find it just as difficult to remember your aunt’s phone number, than what the latest mobile phone looks like. Even though, a phone number should only be a group of 12 numbers. What might hold a clue to this, is the linkage of information within our brains. When one thinks of an English word, we immediately know, how to spell it, what it means, how it sounds, and for some verbs or nouns, an associated image.


From a digital perspective, this approach would be counter-intuitive, since, it would create overhead linking each word to its associated imagery. However, the benefit afforded through this method comes down to how our brains generalize information. When visualizing a complex scene, we can isolate elements within the scene that are stored elsewhere, and therefore do not need to be re-stored. For instance, a bathroom may have a shower, towels, soap, shampoo etc, but a swimming pool washroom may also have similar elements. The human mind can associate the discrete elements with pre-stored generalized objects, thus saving memory. It also re-enforces the importance of the various elements, because if you forget something that is common to one or more contexts, then the context(s) are incomplete.


Some search engines are beginning to use this technique too. If a web page is linked to by many other pages, it is deemed important, and thus prioritized. Information that is not linked is deemed less important. For instance, there are 416,000 links to and none to, thus a search on “book on cats” will show a result from, not


The human brain has evolved from the hunter gatherer days, when it was important to remember and distinguish between poisonous berries, and tasty fruit. In those days there would be little need to remember a set of thirteen digit numbers in order to speak to family members. Thus, our image generalization and storage mechanism is highly evolved, and, although it is simple for our brains to associate a symbolic representation to one of these stored generalised images, our brains find it very difficult to prioritize symbolic information which is not associated with an image, sound, or smell.


Since a phone number is devoid of image, sound, smell, tactile information and so forth, it is weekly linked within our brains, and thus if we do not make a conscious effort to remember it, or refer to it on a daily basis, it will be de-prioritized, and eventually over-written with other information.


Human cache memory is a slightly different issue. This is where rapid-access information is stored temporarily. In this memory, One can remember a phone number long enough to dial it, but not if you are distracted, by another activity, such as talking or writing. This cache memory is investigated in “Conversations with Neil’s brain”, where a patient with a damaged temporal lobe, retained the use of his cache memory, and could recall long-term memories but could no longer store new memories.


Categories: Uncategorized

Configuration files for .NET DLL’s

When performing a HTTP web request against a certain website, I get a protocol violation error. I’ve read that this is caused by a malformed header coming back from the server. (i.e. missing HTTP status tag).

I managed to fix this in code by affing useUnsafeHeaderParsing=true in the web.config, but how do I do this in a dll. there’s no such thing as a .dll.config?

Well, I did find a work around for the protocol violation, by installing VS 2003, .NET 1.1, added an app.config file to the project with useUnsafeHeaderParsing set.


Categories: Uncategorized

casting a OleDbSchemaGuid to a System.GUID

A nice trick I just discovered whilst writing the OleDbWebService (see for details) was that I wanted a function that would encapsulate the GetOleDbSchemaTable function, to which one of the parameters is a OleDbSchemaGuid enumeration. Even though the enumeration’s underlying type is a System.GUID, you can’t make a explicit cast. However – you can cast it to an object and back to a GUID thus !

System.Guid schemaType = (System.Guid)((object)schemaAspect);

Categories: Uncategorized

Unable to find dependency ‘mscorlib’

Getting an unusual warning when building a .NET (in VS .NET 2002) application;

Unable to find dependency ‘mscorlib’ (Signature=’B77A5C561934E089′ Version=’1.0.5000.0′) of assembly ‘System.dll’
Unable to find dependency ‘mscorlib’ (Signature=’B77A5C561934E089′ Version=’1.0.5000.0′) of assembly ‘System.Xml.dll’

Anybody any ideas?


Categories: Uncategorized