Text 21 Dec Converting DBF to CSV files

How to convert DBF to CSV files:

1) Install PERL, DBI, and DBD::XBase (comes as a package for many Linux distributions). 2) Use a script like this one:

#!/usr/bin/perl
use DBI;

my $dbh = DBI->connect("DBI:XBase:/directory/of/dbf/file") or die;
my $sth = $dbh->prepare("select * from YourDbfNameWithoutFileExtension") or die;
$sth->execute() or die $sth->errstr();

# adapt column names
print "Col1Name,Col2Name,Col3Name\n";

my @data;
while (my @data = $sth->fetchrow_array()) {
    print join(",",@data)."\n";
}

3) Call this script and redirect stdout to the CSV file.

Text 11 Dec Android: generate incoming SMS from within your app

(Tested with Android 2.3.3 SDK)

Sometimes, you need to simulate an incoming short message for your app. For instance, an encryption app could need to intercept incoming SMS using a BroadcastReceiver, then check if the SMS was encrypted and if so, decrypt it. Then the decrypted SMS and not the original one should be processed further (show up in SMS app etc.).

The usual way is to create a BroadcastReceiver, intercept the SMS_RECEIVED broadcast on the incoming message and abort the broadcast if necessary (if the message was encrypted or somehow else only for your app). Then you could generate a notifcation and put the SMS into the inbox:

Putting an SMS directy into the native inbox database

You will need the WRITE_SMS privilege.

public static final String SMS_EXTRA_NAME = "pdus";
public static final String SMS_URI = "content://sms";

public static final String ADDRESS = "address";
public static final String PERSON = "person";
public static final String DATE = "date";
public static final String READ = "read";
public static final String STATUS = "status";
public static final String TYPE = "type";
public static final String BODY = "body";
public static final String SEEN = "seen";

public static final int MESSAGE_TYPE_INBOX = 1;
public static final int MESSAGE_TYPE_SENT = 2;

public static final int MESSAGE_IS_NOT_READ = 0;
public static final int MESSAGE_IS_READ = 1;

public static final int MESSAGE_IS_NOT_SEEN = 0;
public static final int MESSAGE_IS_SEEN = 1;

private void putSmsToDatabase( ContentResolver contentResolver, SmsMessage sms )
{
    ContentValues values = new ContentValues();
    values.put( ADDRESS, sms.getOriginatingAddress() );
    values.put( DATE, sms.getTimestampMillis() );
    values.put( READ, MESSAGE_IS_NOT_READ );
    values.put( STATUS, sms.getStatus() );
    values.put( TYPE, MESSAGE_TYPE_INBOX );
    values.put( SEEN, MESSAGE_IS_NOT_SEEN );
    values.put( BODY, sms.getMessageBody().toString() );

    // Push row into the SMS table
    contentResolver.insert( Uri.parse( SMS_URI ), values );
}

Source: http://stackoverflow.com/q/5946262 (modified)

However, this method is not future-safe because the content://sms URI is not specified in the SDK and may be subject to change. Also, if the user uses another SMS app than the default one, the message might not appear in the inbox of this SMS app.

Doing it by generating a SMS_RECEIVED broadcast

So I thought about the possibility to generate a SMS_RECEIVED broadcast by the app. So the message would be processed as a “real” one, including all other filters and then go into the correct SMS app, including all notifications just as from a real one.

A Google research told me that it was not possible: http://stackoverflow.com/q/3819856. I tried myself and after setting the BROADCAST_SMS permission, I didn’t get a permission error but didn’t manage to get it working. Then I found this useful posting: http://d.hatena.ne.jp/thorikawa/20100930/p1 (use a translation service if you don’t speak Japanese – I don’t) and got it working (just call sendSms from your app):

You will need the BROADCAST_SMS privilege.

     private static void sendSms(Context context, String sender, String body) {
             byte [] pdu = null ;
             byte [] scBytes = PhoneNumberUtils
                             .networkPortionToCalledPartyBCD( "0000000000" );
             byte [] senderBytes = PhoneNumberUtils
                             .networkPortionToCalledPartyBCD(sender);
             int lsmcs = scBytes.length;
             byte [] dateBytes = new byte [ 7 ];
             Calendar calendar = new GregorianCalendar();
             dateBytes[ 0 ] = reverseByte(( byte ) (calendar.get(Calendar.YEAR)));
             dateBytes[ 1 ] = reverseByte(( byte ) (calendar.get(Calendar.MONTH) + 1 ));
             dateBytes[ 2 ] = reverseByte(( byte ) (calendar.get(Calendar.DAY_OF_MONTH)));
             dateBytes[ 3 ] = reverseByte(( byte ) (calendar.get(Calendar.HOUR_OF_DAY)));
             dateBytes[ 4 ] = reverseByte(( byte ) (calendar.get(Calendar.MINUTE)));
             dateBytes[ 5 ] = reverseByte(( byte ) (calendar.get(Calendar.SECOND)));
             dateBytes[ 6 ] = reverseByte(( byte ) ((calendar.get(Calendar.ZONE_OFFSET) + calendar
                             .get(Calendar.DST_OFFSET)) / ( 60 * 1000 * 15 )));
             try {
                     ByteArrayOutputStream bo = new ByteArrayOutputStream();
                     bo.write(lsmcs);
                     bo.write(scBytes);
                     bo.write( 0x04 );
                     bo.write(( byte ) sender.length());
                     bo.write(senderBytes);
                     bo.write( 0x00 );
                     bo.write( 0x00 );  // encoding: 0 for default 7bit
                     bo.write(dateBytes);
                     try {
                             byte[] bodybytes  = GsmAlphabet.stringToGsm7BitPacked(body);
                             bo.write(bodybytes);
                     } catch(Exception e) {}

                     pdu = bo.toByteArray();
             } catch (IOException e) {
             }

             Intent intent = new Intent();
             intent.setClassName( "com.android.mms" ,
                             "com.android.mms.transaction.SmsReceiverService" );
             intent.setAction( "android.provider.Telephony.SMS_RECEIVED" );
             intent.putExtra( "pdus" , new Object[] { pdu });
             context.startService(intent);
     }

     private static byte reverseByte( byte b) {
             return ( byte ) ((b & 0xF0 ) >> 4 | (b & 0x0F ) << 4 );
     }

Source: http://d.hatena.ne.jp/thorikawa/20100930/p1 (modified)

I have used GsmAlphabet from the Android source code (I changed all EncodingExceptions to generic Exceptions for testing purposes).

For a good reference of how to create a PDU and what the single fields mean, please have a look at http://www.dreamfabric.com/sms/.

Text 9 Nov How to restart god after deploying a Rails app with Capistrano

If you use the God gem to manage your Ruby on Rails application server (Unicorn, for example), you will have to restart the application server using God after deploying a new version.

I found a solution that polls the application’s tmp directory for restart.txt (like Phusion Passenger does) from within the god configuration and restarts the app if the file is touched. However, it didn’t work well for me because the ps syntax varies on different systems and the method is not very reliable (it triggered when I didn’t want it to, and vice versa).

So I thought about it and now I think that the best solution to restart an app via god is to do (surprise):

god restart YOUR-APP

And it should be done exactly if capistrano deploys, and not by polling.

To allow this command to be run by your deploying user, install sudo and make an entry into sudoers. It may look like this:

thedeployinguser ALL=(root) NOPASSWD: /usr/local/bin/god restart YourRailsServer

Then, all you need to do in config/deploy.rb is to make it look like this:

namespace :deploy do
  task :start do ; end
  task :stop do ; end
  task :restart, :roles => :app, :except => { :no_release => true } do
     run "ln -nfs #{shared_path}/sockets #{release_path}/tmp/sockets"
     sudo "god restart YourRailsServer"
  end
end

(I need the socket symlink for the Unicorn UNIX sockets.)

Text 23 Oct 4 notes [FreeBSD] unicorn.god

I had a very strange problem concerning my Ruby on Rails installation, using FreeBSD, nginx, Unicorn and God.

When starting god from the command line as root, everything worked. However, as soon as I placed the god command into /etc/rc.local or a even service file in /usr/local/etc/rc.d, god didn’t start the Unicorn server but showed exit code 127 for “bundle exec unicorn -E production -c config/unicorn.rb”.

So I added a log directive to the god configuration and looked at the log file. It said:

env: ruby1.9: No such file or directory

Then I changed the god configuration to include /usr/local/bin in the PATH and now everything works. Here my configuration file:

RAILS_ROOT = '/srv/www/my-project-prod/current'

God.watch do |w|
    w.name = 'my-project'
    w.interval = 30.seconds

    w.uid = 'my-project'
    w.gid = 'my-project'

    w.pid_file = File.join(RAILS_ROOT, "tmp/pids/unicorn.pid")
    w.behavior(:clean_pid_file)

    w.dir = RAILS_ROOT
    w.env = { 'PATH' => '/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin' }

    w.start = "bundle exec unicorn -E production -c config/unicorn.rb -D"
    w.stop_signal = 'QUIT'
    w.restart = "kill -USR2 `cat #{w.pid_file}`"
    w.start_grace = w.restart_grace = 30

    w.start_if do |start|
        start.condition(:process_running) do |c|
            c.running = false
        end
    end
end

If somebody is interested, here is my god service file (/usr/local/etc/rc.d/god):

#!/bin/sh
#

# PROVIDE: god
# REQUIRE: DAEMON

. /etc/rc.subr

name="god"
rcvar=`set_rcvar`
command="/usr/local/bin/god"

start_cmd='${command} -c /usr/local/etc/god.conf'
status_cmd='${command} status'
stop_cmd='${command} terminate'

load_rc_config $name
run_rc_command "$1"

Where god_enable=”YES” is set in /etc/rc.conf, and /usr/local/etc/god.conf includes all my project god files.

Text 29 Sep

Anonymous asked: hey, when u installed ts3 on freebsd, did u get the serveradmin password??

yes, it is in the ts3 log file

Text 11 Sep 3 notes Ruby 1.9 invalid multibyte char (US-ASCII) when running bundler via Capistrano on FreeBSD

If your Rails app works fine after upgrading to Ruby 1.9 (including bundler install in development environment), but throws an error like this when you deploy it via Capistrano:

... capistrano runs bundler as deployment user ...
... invalid multibyte char (US-ASCII) ...

make sure that the environment variables are set correctly for this user, especially that the LANG variable sets UTF-8 as charset (for instance, LANG=en_US.UTF-8). Of course, you can just set it in the shell profile of the app user, or you can do it the FreeBSD way in login.conf.

Link 11 Aug On-the-fly RSA encryption for secure transmission of forms/resources over non-secure connections»

I created a new plug-in / gem called acts_as_securely_transferable. It provides on-the-fly RSA encryption for secure transmission of forms/resources over non-secure connections.

See README for more details.

Link 8 Aug Why pay for an SSL certificate when JavaScript does it, too?»

This is a very nice solution to encrypt password fields, credit card numbers etc. in a secure way without needing an expensive SSL certificate. Of course, the authenticity function is lost (i.e. the end user can’t verify who you are, but anybody can buy an SSL certificate too so it doesn’t make much difference). The important thing is the secure transmission of the secret details.

It makes use of this RSA implementation in JavaScript. Don’t store secret information in a CookieStore session (which is used in Rails by default now) - use, for instance, EncryptedCookieStore.

Text 6 Aug Why it doesn’t make any sense to move invalid HTML code into JavaScript

When I added the Google +1 button to one of my Web sites, I noticed that the button code given by Google is not valid HTML 5. There is a HTML 5 version, but when used with i18n (for instance, in German) the JavaScript code is:

<script type="text/javascript" src="https://apis.google.com/js/plusone.js">
  {lang: 'de'}
</script>

Now this isn’t valid HTML 5 because the script tag can have either a source reference (src) or some inline script content.

So I began to search the Web and found solutions like this:

$(function(){
  $('#gplus').append('<g:plusone></g:plusone>');
});

Why doesn’t make this any sense?

Short answer: If you read a newspaper and don’t understand the text - would it be helpful to find instructions how you can write the doubtful text by yourself?

Long answer: Let’s first ask why you want your code to be valid HTML. Probably to get the greatest possible compatibility with current and future browsers and to support the development of the Web. Who do you write the HTML code for?

  1. User agents: Current and future user agents should be able to understand and render your code. In the actual case of the +1 button, user agents may either understand the code and render it or they simply ignore it. There is no need to “hide” the HTML code for them and then insert it by JavaScript because they have to interpret the code inserted by JavaScript nevertheless.

  2. Robots: Normally, they don’t use JavaScript so they would be the target group for “hiding HTML by JS”. However, robots are designed to get most information of the Web and can parse most code that is somehow understandable. In case of correctly opened and closed tags, there is no reason why a robot shouldn’t understand a page that contains an unknown tag. Also, the most important robot is Google and I doubt that Google will reject your page because you embedded the Google +1 button. Furthermore, the actual problem is related to the

  3. For yourself: Do you really feel better when a HTML validator tells you that your code is valid, but you know that it inserts invalid code by JavaScript?

Of course, I have a strong preference for really valid Web code - but moving invalid code from HTML to JavaScript doesn’t make any sense for me.

Text 2 Aug Google: Please give me the -1 button

Where I can vote down (only for me! and maybe Google friends) all these crappy sites that are spamming the search results.


Design crafted by Prashanth Kamalakanthan. Powered by Tumblr.