Stick with the masses ( Eclipse 3.4.1 XPCOM error -2147467262 )

There is a lot to be said about sticking with the masses, going with the crowd so to speak. Especially when you are riding the crowd (crowd) surfing for a different purpose .. huh ?

Let me explain ..

I have been an active user of Linux for the best part of 14 years. In that time, I have sat on the bleeding edge of changes and sometimes suffered. But, also I had time to kill, the fix and fix and patch time was kind of fun.

These days I have a lot less time that I allocate to that discretionary IT time budget, as I have a 2 girls and my beautiful wife with whom I enjoy spending time.

I still mass large amounts of work into my time, but when something goes wrong, ancillary to my task, (or, better put - my tools fail), then i get cranky.

Just on Friday I was prepping to make some changes to camel-file/ftp, adding support for commons-vfs and my eclipse failed to load.

It worked on the Thu, but Fri, nothing. My error said look at workspace/.metadata/.log

In there I find an obscure message...

!ENTRY org.eclipse.ui 4 0 2009-01-12 14:23:58.273
!MESSAGE Unhandled event loop exception
!STACK 0
org.eclipse.swt.SWTException: Failed to execute runnable (org.eclipse.swt.SWTError: XPCOM error -2147467262)

As it turned out, this was caused by an incompatibility with my eclipse and (possible Firefox or something in a mozilla stack).

What was my fix ? Well it turns out, the workspace info page at the first startup of eclipse uses a browser (no worries). But it had fundametal issues launching my system default browser. To get past the bug .. you need to unpack a jar

(/opt/eclipse/plugins/org.eclipse.ui.intro_3.2.201.v20080702_34x.jar)

and edit the plugin.xml, commenting out all extension elements, repack it and restart eclipse with a -clean.

See here for details (this link was the 5th attempt at me getting around the problem).

So why stick with the masses ? Well, I can only guess, but I suspect my running of Ubuntu amd64 is the cause of this eclipse issue biting me. Or, put it mildly, the error has not been tested much on this platform and I got stung, I could be wrong, but I can only guess that, surely others have seen the problem, I am not the only one running eclipse on Linux.

Where it failed for me is that Ubuntu automatically updated (something) and then my eclipse broke. I haven't got the time to work out what updated in the last week due to the auto update manager .. ? question: is there such a place to see the install log.. dpkg must know).

Anyways, my problem is gone and I am back to my usual eclipse-ing self. (minus the intro page, but i have never used that anyway).

bash commands, xml cleaning and Java Searching

A lot of you are Linux users, some Mac and *NIX and the others, I want you to be *NIX. (perhaps use cygwin for a bit).

This post is primarily about two things .. helping you in the command line and helping you develop.

I am, as most people know a Java developer (by trade) and work a lot with XML. I found two things annoyed me for some time a year or two back.

1. Finding the jar of some class I had not seen (and it may be in my m2 repository)
2. Cleaning my XML files (especially the pom.xml files) so they were neat.

I solved both of these with two seperate bash functions. A bash function acts like command on the command line, so if you have




function foo () { echo "foo and Hello World together" }



in your .bashrc, when you "source ~/.bashrc or open another shell, the command "foo" will work.

Cobling this together with some good old unix commandline foo, you can do amazing stuff (I remember the first complex function I wrote was a wrapper for df (disk filesystem) and using awk to show the percentage of usage (of about 2MB) that I had used on the UNE CS Dec Alpha Server, mihi and later neumann).

Sadly I don't have that command anymore, but I don't need it either as df -h gives you the wonderful details. I digress.

Back to functions; it is all well and good to have them, but if you don't remember them, or how to run them, they become dusty (and your self help shell foo rusty).

To solve this problem, I created my own simple help system, "myhelp" nothing robust, nothing fancy but it all works within the one .bashrc file. The principle is simple. Any function I write for me, adheres to two principles.

1. It calls a "help" function first off, which checks if the first argument is --help and promptly displays the help and exits the function.
   
2. After the first { on the same line as the "function ..." brace, i add a comment # myFunction which signifies it is special and help can be listed for it. (and that is is my function)
   

Now to the functions themselves.
1. First the XML Cleaner.

#
# Use XMLLint to reformat the XML
#
function xmlclean () { # myFunction
myShowHelp $1 "xmlclean" " - Reformat the XML Document" || return

doc=$1;
tmpDoc=/tmp/xcleaner.$$.xml
cp ${doc} ${tmpDoc} && xmllint --format ${tmpDoc} --output ${doc} && rm ${tmpDoc}
}

The indentation is a little skewed (I'll try and resolve that later). This bash script uses xmllint

sudo apt-get install libxml2-utils

Fairly simple, in fact I THINK that the xmllint can do an inline replacement, but hey mine works.

so .. formatting an XML file is simply ..

rbuckland@ld630:~$ xmlclean foo.xml

2. Now for locating classes in Jars ..

#
# find a java class in the Repo
#
function jfind () { # myFunction
myShowHelp $1 "jfind" " - Find a classname(substring also) in Maven2 jars. The classname is passed to egrep on the jar contents" || return
find ~/.m2/repository/ -name '*.jar' | xargs -l1 -ixx sh -c "jar tvf xx | egrep $1 && echo xx"
}

An example of it in action:

rbuckland@ld630:~$ jfind DxmlExecutor
1842 Fri Jul 04 09:09:06 EST 2008 com/mivira/dxml/core/DxmlExecutor.class
/home/rbuckland/.m2/repository/com/mivira/dxml/dxml/0.9.1a/dxml-0.9.1a.jar

I won't go into what it is doing (it's pretty simple) just run each command seperately and you will get it all.

3. Now for the help system,

To show your custom commands you have written, type the following.

rbuckland@ld630:~$ myhelp
jfind - Find a classname(substring also) in Maven2 jars. The classname is passed to egrep on the jar contents
xmlclean - Reformat the XML Document
rbuckland@ld630:~$

To see the help for one command ..

rbuckland@ld630:~$ xmlclean --help
xmlclean - Reformat the XML Document

The myhelp works simply by looking for all "function name () { #myFunction" lines in your .bashrc and runs functionName --help for each one.

here is the two help functions you need in the .bashrc.

# declare -F shows the functions, and the shopt 'extdebug' is meant to show the line numbers
# of the functions (and src file) but it didn't so I am going with a more fool proof
# way of identifying my functions .. grep the .bashrc for all function lines that have the
# comment on the same line (hack but works)
#
function myhelp () {
for i in `grep myFunction ~/.bashrc | cut -f2 -d' '`
do
$i --help
done
}

function myShowHelp() {
if [ $1 == '--help' ]; then
echo $2 $3
return 1
fi
return 0
}

That's it. Now when I add a new function. All I have to do is create it according to my template which is

function mynewcommand () { # myFunction
myShowHelp $1 "mynewcommand" "description" || return

# do all the work here
}

and that's it, re-source my .bashrc (source ~/.bashrc) and test it out. and myhelp just automagically picks it up for me.

Done!

Addendum: As is is often the case, looking at my old code I make imporevements and posting this blog entry did just that.

I figured that maven related tasks should be prefixed with m2. and also I added another helper for maven.

Here is my full set of relevant helpers.
Cheers

#-----------------------------------------------------------------------
# declare -F shows the functions, and the shopt 'extdebug' is meant to show the line numbers
# of the functions (and src file) but it didn't so I am going with a more fool proof
# way of identifying my functions .. grep the .bashrc for all function lines that have the
# comment on the same line (hack but works)
#
function myhelp () {
for i in `grep myFunction ~/.bashrc | cut -f2 -d' '`
do
$i --help
done
}

function myShowHelp() {
if [ $1 == '--help' ]; then
echo $2 $3
return 1
fi
return 0
}

#
# print out all the matching jars in the repo
#
function m2showversions () { # myFunction
myShowHelp $1 "m2showversions" " - Simply lists all the jar filenames that match. Infer from that the versions you have on disk"

(cd ~/.m2/repository && find * -name "*$1*")
}

#
# find a java class in the Repo
#
function m2find () { # myFunction
myShowHelp $1 "m2find" " - Find a classname(substring also) in Maven2 jars. The classname is passed to egrep on the jar contents" || return
find ~/.m2/repository/ -name '*.jar' | xargs -l1 -ixx sh -c "jar tvf xx | egrep $1 && echo xx"
}

#
# Use XMLLint to reformat the XML
#
function xmlclean () { # myFunction
myShowHelp $1 "xmlclean" " - Reformat the XML Document" || return

doc=$1;
tmpDoc=/tmp/xcleaner.$$.xml
cp ${doc} ${tmpDoc} && xmllint --format ${tmpDoc} --output ${doc} && rm ${tmpDoc}
}
#-----------------------------------------------------------------------


hope that helps

Hoax Emails - Internet Banking Fraud

In my time of collecting and monitoring SPAM I have noticed that I have not received many Internet Banking Fraud / Hoax emails. It might just be me but it seems a bit odd that the only one that I have received are for Internet Banking sites which I use. So, for example, I use ANZ Internet Banking and just today received my 3rd Internet Banking Hoax Email, this one was for ANZ.

So how do the fraudsters (server registered in Chile (.cl) ) know that I bank ANZ, and why don;t I get NAB, St George or Bank Brunei Email Hoaxes ? The answer is that they KNOW I bank ANZ.

What would be some ways they obtain this list ? I can certainly think of some. And I will add also, ways for the Banks to prevent it, and also for us to be mindful (as users).

Buying the list
At first you would hope it doesn't occur, but when you think about it, it is quite plausible. A high enough price put up entices someone with access to "sell" the list.

Imagine that of 100,000 emails sent out, only 100 people (.001%) provide their details. And if from this, the fraudsters were able to transfer an average of $500, that's a quick easy $50,000.

Of course, some accounts have $50,000 in them to start with, so the bounty is much more of course.

So let's go with $50,000. We can pay $5,000 to the list provider, all they have to provide is Email Addresses of valid customers, nothing more.

$5k is not a lot, but if it were sold for $5k to 3 "fraud" groups, then the insider pockets $15k.

A security breach is 90% of the time an insider job and is the one thing that banks must watch out for.

So who could be an insider ? Well simply put anyone with access to the email list.

This could be:

• Anyone in IT with sufficient privileges (to the test, development, or production database).
• Anyone in Marketing who can extract / export a list of email addresses for bulk email
• Anyone in Customer Servicing with this type of access
• The Vendor of the Banking System who is given the Database in original form for issue resolution (and of course then, any of the vendors employees)

So the insider is an issue. We know it, but it really is as scarey as that. So many IT systems are left open or "available" for reading but all users. I have certainly worked with my fair share that do.

Moving On, so, if it's not an insider job, how else could it be done ?

The ISP Proxy Server
All requests that I make from my ISP (ie: at home) are logged, not only my ISP at home, but at work also.

Might seem odd, but this data is worth money.

At the ISP, they just have to marry my email address and a list of known "banking" logon sites that I access, then they have a decent list. Of course it is not ALL users, but a big ISP (Telstra Bigpond in Australia) would be a nice size and, sadly, the bigger the ISP, the more un-suspecting the user will be and more likely to click on that link.

At a stretch
Virus / URL Logger / Spy
So this one is kind of logical, but probably not done. If there is software on my PC which is "logging" my surfing, they could use that. But if it's on my PC, then well, might as well log the logon to the internet banking site also.

So How do We Prevent it ?
I'll talk about us and the Bank.
For us

(a) Change banks to one which provides
(i) SMS Bank Pay Anyone Verification (With the CommonWealth Bank - not ANZ) for any new Pay Anyone Account not in my list, I need to verify it with an SMS code which is sent to my Mobile phone. That way, I must have my phone with my to pay anyone.

(ii) Secure ID Number Generators for login
HSBC in Australia and HSBC Commercial in the UK use these. A new number is generated every 60 seconds, and I need to supply my ID, password and this number to login.

Fraudsters won't target banks which use either of these (a,i or a,ii) , they either can't get in, or can't do anything once they are in.

(b) NEVER, EVER click on a link in an email for Internet Bank, for changing password, or for reactivating an account, or any activity. Go through your banks website directly, or ring them.

(c) Change banks to someone that DOES provide good access.

For the banks

Where do you start ?

First - Secure internal Access

Perform a full audit of all access methods to the "customer data" at length. This will show holes on how data can come and go, and don't think that just stopping USB access will prevent a user from getting large lists of data out. Where there is a will, there is a way. a quick UUENCODE and a Cut'n'Paste to a pastebin.ca will dump data out in next to no time. If that doesn't seem plausible, what about some steganography ?

Second - Audit Access

Goes without saying, but you would be suprised. Audit all access to the data, so that if data does get out, you know when and by whom (hopefully).

Third - Provide Decent Security Features

It is almost ignorant for banks to NOT have a decent security feature such as SMS verification of Number Generator Fobs for use. They can tell us all they like what NOT to do, but really there are technologies out there which now prevent this type of attack from working, so just get to it an implement it.


There is MUCH more to be said on this topic, but that will get you thinking for now.

Linux - Ubuntu RAID, Modules, my NAS and Fun

Ahh to be minimalist.

It all started over 3 months ago. I have taken to decluttering our house, a personal goal. (I should have set a KPI or two). One task which leads me down this path is digitizing my music and movies.

Many years ago, Sara and I decided to never rent another DVD, but to buy them. We don't buy the latest, unless we "really want it" but will buy the DVD's on special etc, or ex-rentals. Needless to say, from the past 3 years of DVD buying we have a very impressive collection, tipping the 200 mark. Digitising this lot of DVDs has caused a ripple into the Linux world. As you can gather, it takes up lots of space.

To start with, I got rid of some old servers, but kept their hard drives, as my goal is (has been) to setup a home NAS (Network Attached Storage) as the central location for all CDs and DVDs.

So, sitting in front of me is the glorious resurrected "icemaster" (i have to find my chroming again). I have the usual mechano (tip for extending drive bays) and the following drives. 40G x 3, 80G x 3 1 x 160G and 1 x 200G.

My layout is essentially to have a 40G RAID1 / and the rest in LVM as RAID5 devices (split up by like sized partitions).

To have so many drives, of course I needed to extend the IDE channels. (not upgrading to SATA yet.. maybe another few years). I purchased off eBay (new) 2 ITE 8212 IDE RAID cards. The Linux module these use is the it821x or, in the newer Ubuntu 7.10 series, the libata based pata_it821x module. This is where the fun begins.

The card supports RAID0, RAID1, RAID1+0 and JBOD, and just a standard IDE bus. I wanted the latter as I am doing all the RAID in software using the mdadm tools. (very nice).

I have been bouncing around with a few hurdles, and I will detail them each in turn, and how I have solved the, I am sure someone will come up with a similar problem.

So to give a rundown on the hardware we are using.

It's a P4 1.8Ghz, 256M RAM (yes is low spec, but it doesn't need to crunch galaxies, just serve files).
I have a Wireless RT61 based Network Card. (used to always user the rt2x00 serialmonkey module.
There is the standard USB and AGP display card, and an extra 10/100 NIC (8139too if I recall).

Problem #1 - noticing the sd[*] instead of hd[*]
So when I first dived into installing Ubuntu 7.10, I noticed that my IDE harddrives were now appearing as /dev/sd[a-z] drives, instead of the older /dev/hd[a-z] devices. This is now because of the new libata subsystem, supporting PATA and SATA (and SCSI ??) drives. This had a little flow on effect, but largely okay. (just a symantec changed, and I like it actually)

Problem #2 - Recognising the IT8212 PCI RAID Controller.
My first figured I would install onto one of the 40G hard drives, non RAID1 and standard swap of 2G and then setup my RAID and go from there. As I started before the IT8212 cards arrived from Hong Kong (yes it was cheaper then walking to the hardware shop), I actually setup a 150G RAID on booting 40G partition and transferred all my movies and music (and Gigs of data) onto the RAID array. This was on the 2 IDE buses, ie, no CDROM (DVD) after the initial install.

.. then the cards arrived.

The IT8212 by default, in Ubuntu uses the pata_it821x module. In older documentation people have asks some odd questions of this device, but it all seemed to centre around it being used as a RAID controller, ie doing RAID in hardware (good) , but I am using RAID5, which it didn't support.


On first boot, only the primary drives showed up. ie, if the it8212 had a 80G,40G and 200G,160G. Then linux would detect the 80G and 200G only. (odd I thought, must be config).

So into the BIOS of the card, and changed the config to IDE, from RAID (stripe).
Come back out, reboot Linux. no change.

Problem #3 - detected order of the /dev/sd[a-z] changed
Now, I'll pause for a moment from probem #2 and describe #3. when the drives were plugged in, my two boot drives (2 x 40G) on the first IDE bus, were moved from sda and sdb to AFTER the detected drives on the PCI card. For example.

before the IT8212 is plugged in the drives were
/dev/sda (40G)
/dev/sdb (40G)
/dev/sdc (DVD Drive)

After plugging in the card (with the problem #2 still occuring) you see the following

/dev/sda (80G)
/dev/sdb (200G)
/dev/sdc (40G)
/dev/sdd (40G)
/dev/sde (DVD Drive)

Now, the fix to this problem is possibly two areas. One is more robust, the other I had NO success, but new it would work if I persisted.

Solution to Problem #3
When Ubuntu installs, it uses UUID's in the fstab and grub to refer to a harddrive instead of using the known (possibly changing) device location. Let me explain as I see it. A UUID is a Universally Unique Idenitifer and can be created and used as a ID for something. We use them in software / computers all the time. In this case, the Harddrive has a UUID. I don't exactly know who creates them for the drive, but they do exist. Actually, it's the partitions of the disk that have the UUID that we are interested in, the UUID of the harddrive (I assume there is one) is not that important.

So, to make it so it doesn't matter "what" your root drives come under, you use the UUID everywhere to refer to it. that way, when it changes (ie moves, unplugs on the bus etc) it will still be a known device. To locate the uuid for a partition, just use vol_id

once you have the UUID, pop it in the fstab and grub.
/etc/fstab

UUID=897ef-ab67c6-7785-abcb2 / (etc)

and /boot/grub/menu.lst

Now once I had tested that I could (re)boot back in regardless of the "sd-ness" of the drive, I could go back to solving problem #2

A little better (I feel) than the UUID=... is to use LABELs .. this seems to work nicer. But it could clash if you later add a new drive from another system which has a matching label.

So make it easier for me, I just prefixed the label with the short version name of the server. so

icem-root
icem-swap

Getting initramfs to recognise my LABELs (and the UUIDs) was a tad hard.

Solution to Problem #2
Problem #2 was actually simple. and I learnt something too.

The PCI RAID controller had to be told, via the module to load in passthrough mode so it was recognised as just an IDE bus and not a RAID controller.

With the change to libata, the it821x module changed names (and code) to pata_it821x. I read some places about the noraid=1 option for the module. So I tried a few things. Firstly.
if the module (pata_it821x or it821x) is compiled into the kernel, and not as a module, then getting this paratemer into the module is via the kernel parameters (ie, in the /boot/grub/menu.lst)
so .. kernel image ... ro it821x.noraid=1 ...
note the '.' dot in between the noraid and the module name.

I intially got stumped between the pata_ and the (sans pata_) other module.

the pata_ is the newer module, so for Ubuntu 7.10, the it821x doesn't exist (isn't compiled). but I also worked out that the "kernel" line is NOT for modules loaded on the fly.

so .. after I realised that my 7.10 had no it821x, i tried kernel.... pata_it821x.noraid=1
This would have worked had the moduled been compiled into the kernel. not to be.
So .. it was actually meant to be done the way I knew from way back when days.

/etc/modprobe.d/options

option pata_it821x noraid=1

That makes the it821x show the drives on the bus as "just" drives and NOT for linux to detect them as RAID drives. YAY! they all showed up.

Nokia N95 8G, iSync, 3 (Three), music, internet and my Mac

So, recently I have switched over my phone service to 3 (three) for the sole purpose of being a roaming IT worker. Since beginning of October I have found myself without internet access in a lot of places, so I switched to being mobile using three's Internet plans.

To do this, of course it necessitated the phone upgrade (actually, my previous phone was a good 3 year run, the shoicking iMate K-JAM (don't buy)). so .. back to Nokia I went.
The Nokia N95 8GB is the upgraded version to the N95 with the major difference being (for me) all the N95 stuff (software etc) didn't work.

So .. what I wanted was to have this N95 as my modem through bluetooth. To be able to sync calendar, pull photos off it (5MP camera, nice) and to have some stuff to listen to such as my usual Java Posse).

TO do this, seemed simple enough, but there were a few hurdles.

First. Internet access, the most important.
Turns out I had to locate som different modem scripts.
The following was my source of succcess.

<<==-- Internet Access --==>>
Download the Modem Scripts from Ross Barkman (thanks!)
You want the ones labeled Scripts for Nokia 3.5G (HSDPA) phones (26kB): Nokia 3.5G Scripts.

Once you have put them in place ("/Library/Modem Scripts")
you will have three new scripts to use. Nokia HSDPA CID[1-3]

Pair you phone through th Bluetooth preference panel.
Once paired, go into the Network Preference Page.

The following settings are what I have used.

[PPP]
Service Provider:
Account Name: 3netaccess
Password:
Telephone Number:
Alternate Number:

In PPP Options, the following are "ticked"

[v] Disconnect when user logs out
[v] Disconnect when switching user accounts
[v] Redial if busy
[v] Send PPP echo packets
[v] Use TCP header compression

[TCP/IP]
Configure IPv4: [Using PPP]

[Proxies]
** no proxies configured (i don't use any)

[Bluetooth Modem]
Nokia HSDPA CID1

I have unticked the following

[ ] Enable error connection and compression in modem
[ ] Wait for dial tone before dialling

Now that that is all done, see if you can connect :-) the "3netaccess" account name, I found that from the following link

<<==-- iSync--==>>
So, Internet Access was probably the easiest (1/2 hour research).
iSync proved to be a good week battle of trying many iSync plugins until I happened upon the only one that worked which was this.

iSync Plugin for Nokia N95 8GB

Thank you Jussi Edlund!

With this iSync plugin installed iSync works.

<<==-- Music and Pictures--==>>
So, Nokia, they have this "almost" right .. the camera appears in iPhoto, but the phone doesn't appear as an iTunes "player" (probably can't do that).
But anyway .. from Europe, download the following ..

Nokia Multimedia Transfer

Hope that all helps someone!
Have a great day.