Sending raw XML with php nusoap client

In my last blog post I described how to use wsdl2php to parse a wsdl file and generate php classes. This should make your life a lot simpler when dealing with SOAP Web Services.

But there are times that all the frameworks and toolkits in the world wont be able to produce the desired plain XML structure of the SOAP call. Especially when you are working with Chinese low-end equipment.

I usually work in php nusoap and I had trouble finding the right way to do this. I was so desperate, that I have been posting raw XML with CURL to mimic the SOAP Request. Luckily I found a way to do i with our favourite php soap toolkit.

Thow following is an example that calls a function insertUser with raw XML:


require_once('nusoap/lib/nusoap.php');

$endpoint = "http://yoursoapserver.com/SomeSoapService";

$client = new nusoap_client($endpoint, false);

$msg = $client->serializeEnvelope("
<ns1:insertuser xmlns:1="http://yoursoapserver.com/">
  <arg0 xsi:type="xsd:string">name</arg0>
  <arg1>
    <username xsi:type="xsd:string">username</username>
    <password xsi:type="xsd:string">0ac495f743a36cef9b0eaafa92ae08e21</password>
  </arg1>
  <arg2 xsi:type="xsd:string">email</arg2>
  <arg3 xsi:type="xsd:string">domain</arg3>
</ns1:insertuser>
");

$result=$client->send($msg, $endpoint);

print_r($result);

Mysql database migration and character set conversion

How often have you stumbled on a mysql dump file with strange hieroglyphs. You have an old Mysql 4 database in latin1 character set and you need to migrate it on a new server with utf8 support.

The procedure is simple and easy as long as you know the steps.

The first thing you need to do is make a mysql dump with the flag --default-characer-set set as the charset of the source db:

mysqldump --default-character-set=latin1 -u user -p old_db > dump.file

Now you need to replace all inctances of the old charset (latin1) in the dumpfile with the newcharset (utf8). You can do this with sed from the command line

sed "s/latin1/utf8/" dump.file > new_dump.file

Create the new database with the desired charset on the new server with the SQL command:

CREATE DATABASE new_db CHARSET utf8 COLLATE utf8_general_ci

And finaly restore the database with the command

cat new_dump.file | mysql --default-character-set=utf8 -u user -p newdb

Afterwards be sure too make all sql connections with utf8 encoding. You can execute this sql command imminently after you make a the connection to the db from the application:

SET NAMES utf8

And that is all there is to it.

Automate web clicks in perl with WWW::Mechanize

I have been struggling a lot lately with incomplete API's to systems that I depend on. Badly documented XML's, incomplete SOAP services, complex databases. The idea to automate the click-able processes is not a genuine one, but when you don't have a proper interface it becomes a painstaking effort. As I am a php native, I was struggling a lot with CURL, until I found a comment on a webforum to try perl.

Enter WWW::Mechanize.

In order to use this module you need to install it first, the best way being from perl's cpan:

cpan> install WWW::Mechanize

After the installation is complete u can use it in your perl script by adding:

use WWW::Mechanize;

The first thing you need to do is get the web page where the desired form is located. You write

my $mech = WWW::Mechanize->new();
$mech->get("http://somesite.com/index.php");

After you get the web page you would usualy like to submit a form. You can chose the form you want to submit in two ways by name or by number. You need to specify the fields that you want to submit, and the submit button. These two options are optional and can be omitted.

Lets say you need to login somewhere:

$result = $mech->submit_form(
form_name => 'frmLogin', #name of the form
#instead of form name you can specify
#form_number => 1
fields      => 
{
 txtUser    => 'admin', # name of the input field and value 
 txtPass    => 'adminpass',
}
button    => 'btnSubmit' #name of the submit button
);

Retrieving server reply

After you log in, $mech will keep the session data and you can continue to the next step. In case you need to see what did the web site reply after you submitted the form, you can print out the the returned html with:

print $result->content();

You might also need this if you need to parse the returned html.

Multi-step automation


In case you have to complete several steps, don't worry just continue on as with the first step. Mech keeps the last returned html web page, so you can go on and submit a second form from with the same procedure:

#STEP 1: Login
$mech->submit_form(
form_name => 'frmLogin', 
fields      => 
{
 txtUser    => 'admin', 
 txtPass    => 'adminpass',
},
button    => 'btnSubmit' 
);

#STEP 2: Input form
$mech->submit_form(
form_name => 'frmInput', 
fields      => 
{
 txtField    => 'some value', 
 selectbox    => 'another value',
},
button    => 'btnSubmit' 
);

Beware of JavaScript


The only drawback with WWW::Mechanize is that it does not support javascript. To bad as most modern web pages have at least some client side scripting. In case you cant go pass the javascript, try to figure out what it is doing, and try to simulate it. Ussually a simple javascript will populate some select boxes or set some hidden fields. Always try to find what are the exact values that you need to submit, and go with it.

Examples

In addition you can find a complete example of simple form submission. The script connects to a network device's web interface and reboots it. It takes a single argument, the ip of the device. It is executed from the command line with

perl reboot_device.pl 192.168.1.1

#!/usr/bin/perl -w
use WWW::Mechanize;

my $ip = $ARGV[0];
 
my $mech = WWW::Mechanize->new();

$url_logon = "http://$ip:8080";
$url_reboot= "http://$ip:8080/reboot.htm";

$mech->get($url_logon);
$mech->submit_form(
form_number => 1,
fields      => 
{
 user    => 'admin',
 pass    => 'adminpass',
}
);

##### STEP 1: DATES #####
$mech->get( $url_reboot );
$mech->submit_form(
form_number => 1,
button    => 'submit'
);

Working around

Working around is one of the main job descriptions of an ambitious developer. Especially when you are a third-world ambitious developer in a company with low-budget high expectations.

Actually I am not here to complain. I love working around. I love developing interesting and clever solutions that solve hopeless and expensive problems. My whole carer is based on workarounds, and I would like to share it. Because sharing is caring.

My favorite part about workarounds is the political aspect. When a simple hack saves your company X thousand of dollars that you can distribute as a bonus to your employees, it goes way beyond technology and economics. It becomes political, as you take the goodies away from the big players, and give it to the community. Kinda like Robin Hood with geeky glasses.

This is my opening blog post. What I plan to do in the future is share my experience and keep it simple. Until then, I will be working around.