About to embark on something ambitiuos (I suspect), I figured I would detail through some blog posts how I go.
I am building an application (secret squirrel until it releases) that I want Scala to be it's core language.
The GUI I want in Java (it just works so well), Eclipse is just the default IDE for me, Maven my absolute must build tool and then all the goodness of Spring and Hibernate as usual.
My first challenge (beyond learning Scala) is to get Eclipse, Maven and Scala to co-operate. The project will have Java source and Scala Source mixed in.
GWT, as per my other standard projects will be in it's own module anyway.
So.. here goes. This is just the "I am trying this" post. I'll let you all know how I go.
Read on for Part 2
Showing posts with label gwt. Show all posts
Showing posts with label gwt. Show all posts
Saturday, 20 March 2010
Thursday, 2 August 2007
GWT Security Concerns - Object Injection
GWT opens a new way of building webapps with it's different AJAX development method(ology) and it also opens a new area of "hackable" software.
I will term this new area "Object Injection".
GWT makes the AJAX development easier as the code is written in a type safe language (Java) and "compiled" out to JavaScript, dealing with all the browser inconsistencies and nuances behind the scenes. You still have the ability to write JavaScript via a Java Native Interface (JNI) and "look and feel" is done through CSS.
The process of writing a GWT web application is to write your GUI web interface or just parts of it in Java , and using the GWT compiler (a java app) compile your Java to JavaScript.
What the GWT does, is trawls your code and converts what it finds to JavaScript. So if you use ObjectA as a POJO to make changes to a Drop Down Select Box (for example) then this ObjectA will be exposed (someway) as JavaScript.
One of the niceties that the GWT Java-to-JavaScript model brings up is that your direct domain model objects can be worked on at the front interface without the need of DTOs, a code/time saving benefit.
So .. code might look something like (simplified by not showing the Asyn Callback()
MyDomainObejct obj = finder.findDOByID(myId);
myTextBox.setText(obj.getName());
Now this code above would be compiled to JavaScript and, even with the GWT J2JS compiler set to "DETAILED" and not OBF(USCATED)" you get some pretty hefty JavaScript.
This is some real JavaScript from the GWT in Detailed Mode, showing a "BusinessEntity" object. (a POJO if you like)
function pkg_BusinessEntity(){
}
_ = pkg_BusinessEntity.prototype = new java_lang_Object();
_.java_lang_Object_typeName = package_pkg_ + 'BusinessEntity';
_.java_lang_Object_typeId = 4;
_.pkg_BusinessEntity_abn = null;
_.pkg_BusinessEntity_commencementDate = null;
_.pkg_BusinessEntity_isCurrent = false;
_.pkg_BusinessEntity_isRegisteredForGST = false;
_.pkg_BusinessEntity_name = null;
_.pkg_BusinessEntity_postcode = null;
_.pkg_BusinessEntity_registeredState = null;
So, where is the security concern ?
Well in some cases, such as most of my world, the domain model is fairly significant, with trading / transaction objects and user accounts with passwords, and money and settings etc etc.
So if, in a GWT class, someone were simply to expose the DAO (Data Access Object) service such as
service.storeObject(myUserObject);
Then, essentially we have, assuming there is no or not sufficient checks, a direct way to store objects into the domain (database) from the browser.
I will call this "Object Injection" in the same vain as "SQL Injection". An Object injection hole is could really be termed as a
Poorly Coded RPC interface which does not check values and who and why of what it is "doing".
Of course, finding an Object Injection hole in a GWT application is going to prove fairly complex to do (but surely not as hard as searching for a buffer over/under run). The types of tasks that a would-be inquisitive person might have to do is as follows.
Anything you do, can be repeated, out of step with simple JavaScript calls.
If parts of the domain model are exposed, this is a good place to start hacking.
The problem is no different with current non-GWT AJAX development. It is just that GWT does a lot of smoke and mirror compiling for you and if you unwittingly create holes in your Java RPC methods (without sufficient checks), then those holes are exposed for all to see (and play with).
So what techniques can we use to prevent us making such a holes. The following would be my quick and dirty set of rules.
1. Don't expose the domain model to the GWT compiler.
Why ? Well for a few reasons
(a) It will expose your internal workings of the application to the browser, and this is one place where the would-be nefarious person can start looking to analyse your application to understand its structure, thus weaknesses. You might think, "Gee, that's a lot of work when this JavaScript is obfuscated", but think for a second, is there anywhere in you application that one can profit (monetarily or otherwise) from your application, giving them an incentive to start the work ?
(b) To expose a Domain Model Object, means you probably want to "change" and store it, which means the DAO service (or similar) will have to take an object and save it to the database. Simply, find that DAO Service and the right object that it requires, and I can then start poking in (Object Injection) other object values to see if I change some stuff at the server.
2. Have Users / Accounts and Password - and Audit them
If you use usernames/passwords and sufficient checks in the right places you will have the ability to log who is doing what. Another way to think of this is, minimise the amount of GWT you use to people who are not authenticated against your application.
A way that I always achieve this these days is to use Acgei Security with Spring and webfilters over the whole application. This way I know which users are hitting which RPC methods and could audit appropriately.
3. Expose ONLY that which needs to be - Use DTO's, simple APIs, push "work" to the server, Separate Projects.
Use DTO's to push and pull your data back and forth. This way you can check the DTO values before storing and also know what is being exposed at the front end.
Use Simple API's - Make you RPC methods simple (ie, don't push too much back and forth). This is probably just a KISS principal and will help in making sure you don't inadvertantly expose that which shouldn't be.
Move as much logic as you can to the server. This ties in with the previous point. A simple API logically means that the complex stuff is on the server. This way, code which does not need to be seen, is not seen. Plus the burden on the browser *our unknown environment* is kept at a minimum (which has an added bonus because we don't want the help desk of our application getting complain calls saying the application slows down their browser thus tarnishing the "company".)
Separate Projects - One method I am using in my GWT apps is to have a separate project for GWT compiled code, for server (RPC and JSPs etc) and business / domain model. I use Maven 2 and a parent project compiles and assembles all 3 pieces together. So three projects, but ONLY one is the GWT compiled code, so I know that, anything potentially that goes in this client project, is going to be in JavaScript. Anything OUTSIDE that project, is hidden from the browser.
4. Secure all RPC interfaces
Use Aspects or Filters (I use Acegi Security) to secure your GWT RPC methods. You will be able to lock down the method calls to the user and also ensure that the "objects" are allowed to be transported that are.
I hope this little entry helps people see the potential issues and raise awareness in their designs about this potential security "feature".
It is nothing new, of course, but the way in which GWT development is done, can have an impact on the "bad applications" we write if we are not careful.
I will term this new area "Object Injection".
GWT makes the AJAX development easier as the code is written in a type safe language (Java) and "compiled" out to JavaScript, dealing with all the browser inconsistencies and nuances behind the scenes. You still have the ability to write JavaScript via a Java Native Interface (JNI) and "look and feel" is done through CSS.
The process of writing a GWT web application is to write your GUI web interface or just parts of it in Java , and using the GWT compiler (a java app) compile your Java to JavaScript.
What the GWT does, is trawls your code and converts what it finds to JavaScript. So if you use ObjectA as a POJO to make changes to a Drop Down Select Box (for example) then this ObjectA will be exposed (someway) as JavaScript.
One of the niceties that the GWT Java-to-JavaScript model brings up is that your direct domain model objects can be worked on at the front interface without the need of DTOs, a code/time saving benefit.
So .. code might look something like (simplified by not showing the Asyn Callback()
MyDomainObejct obj = finder.findDOByID(myId);
myTextBox.setText(obj.getName());
Now this code above would be compiled to JavaScript and, even with the GWT J2JS compiler set to "DETAILED" and not OBF(USCATED)" you get some pretty hefty JavaScript.
This is some real JavaScript from the GWT in Detailed Mode, showing a "BusinessEntity" object. (a POJO if you like)
function pkg_BusinessEntity(){
}
_ = pkg_BusinessEntity.prototype = new java_lang_Object();
_.java_lang_Object_typeName = package_pkg_ + 'BusinessEntity';
_.java_lang_Object_typeId = 4;
_.pkg_BusinessEntity_abn = null;
_.pkg_BusinessEntity_commencementDate = null;
_.pkg_BusinessEntity_isCurrent = false;
_.pkg_BusinessEntity_isRegisteredForGST = false;
_.pkg_BusinessEntity_name = null;
_.pkg_BusinessEntity_postcode = null;
_.pkg_BusinessEntity_registeredState = null;
So, where is the security concern ?
Well in some cases, such as most of my world, the domain model is fairly significant, with trading / transaction objects and user accounts with passwords, and money and settings etc etc.
So if, in a GWT class, someone were simply to expose the DAO (Data Access Object) service such as
service.storeObject(myUserObject);
Then, essentially we have, assuming there is no or not sufficient checks, a direct way to store objects into the domain (database) from the browser.
I will call this "Object Injection" in the same vain as "SQL Injection". An Object injection hole is could really be termed as a
Poorly Coded RPC interface which does not check values and who and why of what it is "doing".
Of course, finding an Object Injection hole in a GWT application is going to prove fairly complex to do (but surely not as hard as searching for a buffer over/under run). The types of tasks that a would-be inquisitive person might have to do is as follows.
- De-Obfuscate the javascript to find the "meaningful" objects that are being worked with
- Dump out all the "Service" methods and try and tie these to where they are being used in the GUI (Web Application).
- Identify the types of objects which would provide benefit of "changing", such as Bank Accounts, Limits, restrcitions, user account saving (like storing settings etc) and locate how the interface is working with the RPC method calls
- Write some Javascript to create a generated GWT object .. and send it to the RPC API.
Anything you do, can be repeated, out of step with simple JavaScript calls.
If parts of the domain model are exposed, this is a good place to start hacking.
The problem is no different with current non-GWT AJAX development. It is just that GWT does a lot of smoke and mirror compiling for you and if you unwittingly create holes in your Java RPC methods (without sufficient checks), then those holes are exposed for all to see (and play with).
So what techniques can we use to prevent us making such a holes. The following would be my quick and dirty set of rules.
1. Don't expose the domain model to the GWT compiler.
Why ? Well for a few reasons
(a) It will expose your internal workings of the application to the browser, and this is one place where the would-be nefarious person can start looking to analyse your application to understand its structure, thus weaknesses. You might think, "Gee, that's a lot of work when this JavaScript is obfuscated", but think for a second, is there anywhere in you application that one can profit (monetarily or otherwise) from your application, giving them an incentive to start the work ?
(b) To expose a Domain Model Object, means you probably want to "change" and store it, which means the DAO service (or similar) will have to take an object and save it to the database. Simply, find that DAO Service and the right object that it requires, and I can then start poking in (Object Injection) other object values to see if I change some stuff at the server.
2. Have Users / Accounts and Password - and Audit them
If you use usernames/passwords and sufficient checks in the right places you will have the ability to log who is doing what. Another way to think of this is, minimise the amount of GWT you use to people who are not authenticated against your application.
A way that I always achieve this these days is to use Acgei Security with Spring and webfilters over the whole application. This way I know which users are hitting which RPC methods and could audit appropriately.
3. Expose ONLY that which needs to be - Use DTO's, simple APIs, push "work" to the server, Separate Projects.
Use DTO's to push and pull your data back and forth. This way you can check the DTO values before storing and also know what is being exposed at the front end.
Use Simple API's - Make you RPC methods simple (ie, don't push too much back and forth). This is probably just a KISS principal and will help in making sure you don't inadvertantly expose that which shouldn't be.
Move as much logic as you can to the server. This ties in with the previous point. A simple API logically means that the complex stuff is on the server. This way, code which does not need to be seen, is not seen. Plus the burden on the browser *our unknown environment* is kept at a minimum (which has an added bonus because we don't want the help desk of our application getting complain calls saying the application slows down their browser thus tarnishing the "company".)
Separate Projects - One method I am using in my GWT apps is to have a separate project for GWT compiled code, for server (RPC and JSPs etc) and business / domain model. I use Maven 2 and a parent project compiles and assembles all 3 pieces together. So three projects, but ONLY one is the GWT compiled code, so I know that, anything potentially that goes in this client project, is going to be in JavaScript. Anything OUTSIDE that project, is hidden from the browser.
4. Secure all RPC interfaces
Use Aspects or Filters (I use Acegi Security) to secure your GWT RPC methods. You will be able to lock down the method calls to the user and also ensure that the "objects" are allowed to be transported that are.
I hope this little entry helps people see the potential issues and raise awareness in their designs about this potential security "feature".
It is nothing new, of course, but the way in which GWT development is done, can have an impact on the "bad applications" we write if we are not careful.
Wednesday, 25 July 2007
GWT Fun
Over the past 4 weeks I have been building an impressive list of what to do an what not to do when it comes to GWT. In my last post I mentioned an issue with JSP 2.0 and GWT.
Tip #1
Tonight, I came across a very similar issue with IE6. (might be IE7 problem also).
the Script tag which you include the GWT module with looks like
<script .../>
But , if that is how you do it, IE won't like it. In fact it will render a big blank page with no error and "Done." down the bottom.
You won't even see an ounce of the "page" even if GWT is not the primary content.
The quick fix is to change the single script tag to have an open and close (and put some space between them).
That took all of about 20 minutes to discover.
So far, my verdict on GWT is 80% for and 20% against. I am certain the 20% against, ie things that I can find I don't like, are because I have not yet worked out the "good" or proper way of doing things.
Tip #2
Some people have mentioned about needing the rpc interfaces in the client package and that they don;t like it. Well I didn't like it either. I have an rpc package.
I found a sneaky (smelly) way of just having arbitrary "code" for GWT to find.
Create a "BaseGWT.gwt.xml" and point the entry-point class at an empy implementation of EntryPoint. But in this gwt.xml, set all the source folders that you want.
Then in the "other" modules, inherit it. Now, the compiler (maven2) complains and errors, but because of a bug in the maven2 GWT plugin, it keeps going (and doesn't fail) and the other modules happily inherit this dummy GWT module and all the trappings (other source folders).
When I find a better way of doing this, I'll holler.
Tip #1
Tonight, I came across a very similar issue with IE6. (might be IE7 problem also).
the Script tag which you include the GWT module with looks like
<script .../>
But , if that is how you do it, IE won't like it. In fact it will render a big blank page with no error and "Done." down the bottom.
You won't even see an ounce of the "page" even if GWT is not the primary content.
The quick fix is to change the single script tag to have an open and close (and put some space between them).
That took all of about 20 minutes to discover.
So far, my verdict on GWT is 80% for and 20% against. I am certain the 20% against, ie things that I can find I don't like, are because I have not yet worked out the "good" or proper way of doing things.
Tip #2
Some people have mentioned about needing the rpc interfaces in the client package and that they don;t like it. Well I didn't like it either. I have an rpc package.
I found a sneaky (smelly) way of just having arbitrary "code" for GWT to find.
Create a "BaseGWT.gwt.xml" and point the entry-point class at an empy implementation of EntryPoint. But in this gwt.xml, set all the source folders that you want.
Then in the "other" modules, inherit it. Now, the compiler (maven2) complains and errors, but because of a bug in the maven2 GWT plugin, it keeps going (and doesn't fail) and the other modules happily inherit this dummy GWT module and all the trappings (other source folders).
When I find a better way of doing this, I'll holler.
Saturday, 9 June 2007
GWT and Maven 2, OH the pain!
GWT - Maven2 and Eclipse .. Ouch!
Using GWT for all of 3 weeks now, I am finding it is nice. It's logical, well thought out and simple.
But, I think there is a little way to go when it comes to using some de jure standards. I am a big Maven 2 fan because in an Enterprise (captial E for 'ooh' fancy) environment, standards go a LONG way and Maven 2 dictates a nice and well understood way of doing things.
So, what's me beef with GWT ? Simply this, gosh it has been hard to get it working with Maven 2.
Now, part of this is because I am still trying to get my head around how it works. I have been using the applicationCreator script to setup a quick hello world sample, then move that into the src/main/java and src/main/resources folders.
But what does that do to my src/main/webapp ? where do the images get pulled from ? why don't index.jsp load in the GWTShell (when it's tomcat underneath) ? So many questions.
So, there is a maven 2 plugin. I am currently using the 1.5.2 version of it.
Google Code Homepage : http://code.google.com/p/gwt-maven/
Maven 2 Plugin Doc : http://gwt-maven.googlecode.com/...
A list of some problems : http://code.google.com/p/gwt-maven/wiki/FAQ
Maven Repo : http://gwt-maven.googlecode.com/...
SVN Repository : http://gwt-maven.googlecode.com/.../
and as it turns out, there are a few bugs and features which need some work. (I wish I saw that FAQ page first before diving in)
The first issue I cam across was the "plural" vs "singular" naming convention. In Maven 2, a configuration item for a plugin that requires more than one value is pluralised .. ie
<options>
<option>...</option>
<option>...</option>
</options>
What's the problem ?
Well, the com.totsp.gwt plugin needs a configuration line like
<compiletarget>org.sobbo.ui.Home</compiletarget>
However, this config value in the plugin source needs an [] array or targets. (logical yes) but m2 plugins, dictate that the property name is then plural (compileTargets) otherwise you get this kind of error message.
[INFO] Failed to configure plugin parameters for: com.totsp.gwt:maven-googlewebtoolkit2-plugin:1.5.3-SNAPSHOT
(found static expression: 'org.sobbo.ui.client.Home' which may act as a default value).
Cause: Cannot assign configuration entry 'compileTarget' to 'class [Ljava.lang.String;' from 'org.sobbo.ui.client.Home', which is of type class java.lang.String
so .. the AbstractGWTMojo has to change to have a plural of configurationTargets (there must be a way else how would they be using it right now ?
It turns out there is an issue logged for this .. http://code.google.com/p/gwt-maven/issues/detail?id=37
Anyways, what's the next pain ? well to checkout the src from the subversion repository, you get this nice little gem.
$ svn co http://gwt-maven.googlecode.com/svn/trunk/maven-googlewebtoolkit2-plugin/
svn: REPORT request failed on '/svn/!svn/vcc/default'
svn: REPORT of '/svn/!svn/vcc/default': 400 Bad Request (http://gwt-maven.googlecode.com)
BANG, and there is my pain. This looks like a straight up Google SVN problem. Funny enough, I have an SVN downloaded script that works to "download" src from SVN repositories using wget.
See the end of this post for the shell script.
So .. my third issue ? Well there was some unusual "Java Execution Mojo Bootstrapping" stuff going on, and it's broken. Essentially when trying to launch java to compile (testing, shell etc) it (the GWT maven plugin) couldn't find it (java) and died. I found the problem (after downloading the src and corrected it) and logged this issue with a patch @ GWT M2 Plugin Issue 43
So .. is it ready ? This Maven 2 plugin really needs to be working, maven 2 is big and GWT / M2 / Spring / Eclipse is pretty important (to me at least).
I'll keep battling with it because this way I can probably make it a little better. I hope that it gets better.
So, that SVN downloader script, it's not completely "safe" because the revision could change by a third party part way through your download, but hey, it works.
#!/bin/sh
# Author: Ramon Buckland ramon#at#thebuckland.com
# Quick hack script to pull the latest SVN Revision version from a repo repo when you have no SVN tools
# or SVN through ISA proxy servers are just not working ..
#
function usage {
echo "$0 "
echo "svn-url: SVN URL is a URL to the trunk or a tag you interested in"
echo " eg: http://svn.apache.org/repos/asf/incubator/servicemix/trunk/"
echo "product-name: A short name of the product so that we can create a directory for you"
echo " eg: servicemix"
echo "(also, set the http_proxy=http://hostrunning-ntlmaps:port)"
exit
}
if [ "X$1" == 'X' ]; then
usage
fi
if [ "X$2" == 'X' ]; then
usage
fi
if [ "X$http_proxy" == 'X' ]; then
echo "WARNING: http_proxy is not set. Do you need it ?"
fi
SVN_REPO=$1
PRODUCT_NAME=$2
mkdir ${PRODUCT_NAME}-svn-pid-$$
cd ${PRODUCT_NAME}-svn-pid-$$
# nv=non-verbose
# nH=noHost directory created
# -np=no ascend to paremt dirs
# --cut-dirs=3
# -erobots=off .. don't look at robots.txt to see what you are and aren't allowed to do
# -m mirror
wget -nv -nH -np --cut-dirs=3 -erobots=off -m ${SVN_REPO}
NEWDIR=`grep Revision index.html | grep h2 | cut -d':' -f1 | cut -d'>' -f2 | tr ' ' _`
cd ..
mv ${PRODUCT_NAME}-svn-pid-$$ ${PRODUCT_NAME}-svn-${NEWDIR}
find . -type f -name index.html | xargs rm
Hope that helps someone. I use it a bit here and there.
Using GWT for all of 3 weeks now, I am finding it is nice. It's logical, well thought out and simple.
But, I think there is a little way to go when it comes to using some de jure standards. I am a big Maven 2 fan because in an Enterprise (captial E for 'ooh' fancy) environment, standards go a LONG way and Maven 2 dictates a nice and well understood way of doing things.
So, what's me beef with GWT ? Simply this, gosh it has been hard to get it working with Maven 2.
Now, part of this is because I am still trying to get my head around how it works. I have been using the applicationCreator script to setup a quick hello world sample, then move that into the src/main/java and src/main/resources folders.
But what does that do to my src/main/webapp ? where do the images get pulled from ? why don't index.jsp load in the GWTShell (when it's tomcat underneath) ? So many questions.
So, there is a maven 2 plugin. I am currently using the 1.5.2 version of it.
Google Code Homepage : http://code.google.com/p/gwt-maven/
Maven 2 Plugin Doc : http://gwt-maven.googlecode.com/...
A list of some problems : http://code.google.com/p/gwt-maven/wiki/FAQ
Maven Repo : http://gwt-maven.googlecode.com/...
SVN Repository : http://gwt-maven.googlecode.com/.../
and as it turns out, there are a few bugs and features which need some work. (I wish I saw that FAQ page first before diving in)
The first issue I cam across was the "plural" vs "singular" naming convention. In Maven 2, a configuration item for a plugin that requires more than one value is pluralised .. ie
<options>
<option>...</option>
<option>...</option>
</options>
What's the problem ?
Well, the com.totsp.gwt plugin needs a configuration line like
<compiletarget>org.sobbo.ui.Home</compiletarget>
However, this config value in the plugin source needs an [] array or targets. (logical yes) but m2 plugins, dictate that the property name is then plural (compileTargets) otherwise you get this kind of error message.
[INFO] Failed to configure plugin parameters for: com.totsp.gwt:maven-googlewebtoolkit2-plugin:1.5.3-SNAPSHOT
(found static expression: 'org.sobbo.ui.client.Home' which may act as a default value).
Cause: Cannot assign configuration entry 'compileTarget' to 'class [Ljava.lang.String;' from 'org.sobbo.ui.client.Home', which is of type class java.lang.String
so .. the AbstractGWTMojo has to change to have a plural of configurationTargets (there must be a way else how would they be using it right now ?
It turns out there is an issue logged for this .. http://code.google.com/p/gwt-maven/issues/detail?id=37
Anyways, what's the next pain ? well to checkout the src from the subversion repository, you get this nice little gem.
$ svn co http://gwt-maven.googlecode.com/svn/trunk/maven-googlewebtoolkit2-plugin/
svn: REPORT request failed on '/svn/!svn/vcc/default'
svn: REPORT of '/svn/!svn/vcc/default': 400 Bad Request (http://gwt-maven.googlecode.com)
BANG, and there is my pain. This looks like a straight up Google SVN problem. Funny enough, I have an SVN downloaded script that works to "download" src from SVN repositories using wget.
See the end of this post for the shell script.
So .. my third issue ? Well there was some unusual "Java Execution Mojo Bootstrapping" stuff going on, and it's broken. Essentially when trying to launch java to compile (testing, shell etc) it (the GWT maven plugin) couldn't find it (java) and died. I found the problem (after downloading the src and corrected it) and logged this issue with a patch @ GWT M2 Plugin Issue 43
So .. is it ready ? This Maven 2 plugin really needs to be working, maven 2 is big and GWT / M2 / Spring / Eclipse is pretty important (to me at least).
I'll keep battling with it because this way I can probably make it a little better. I hope that it gets better.
So, that SVN downloader script, it's not completely "safe" because the revision could change by a third party part way through your download, but hey, it works.
#!/bin/sh
# Author: Ramon Buckland ramon#at#thebuckland.com
# Quick hack script to pull the latest SVN Revision version from a repo repo when you have no SVN tools
# or SVN through ISA proxy servers are just not working ..
#
function usage {
echo "$0
echo "svn-url: SVN URL is a URL to the trunk or a tag you interested in"
echo " eg: http://svn.apache.org/repos/asf/incubator/servicemix/trunk/"
echo "product-name: A short name of the product so that we can create a directory for you"
echo " eg: servicemix"
echo "(also, set the http_proxy=http://hostrunning-ntlmaps:port)"
exit
}
if [ "X$1" == 'X' ]; then
usage
fi
if [ "X$2" == 'X' ]; then
usage
fi
if [ "X$http_proxy" == 'X' ]; then
echo "WARNING: http_proxy is not set. Do you need it ?"
fi
SVN_REPO=$1
PRODUCT_NAME=$2
mkdir ${PRODUCT_NAME}-svn-pid-$$
cd ${PRODUCT_NAME}-svn-pid-$$
# nv=non-verbose
# nH=noHost directory created
# -np=no ascend to paremt dirs
# --cut-dirs=3
# -erobots=off .. don't look at robots.txt to see what you are and aren't allowed to do
# -m mirror
wget -nv -nH -np --cut-dirs=3 -erobots=off -m ${SVN_REPO}
NEWDIR=`grep Revision index.html | grep h2 | cut -d':' -f1 | cut -d'>' -f2 | tr ' ' _`
cd ..
mv ${PRODUCT_NAME}-svn-pid-$$ ${PRODUCT_NAME}-svn-${NEWDIR}
find . -type f -name index.html | xargs rm
Hope that helps someone. I use it a bit here and there.
Subscribe to:
Posts (Atom)