A Dev Blog

[Typo3 tt_news] How to fix "No news in this list"

Sun, 12 Feb 2012 02:04:49 +0100

[Typo3 tt_news] How to fix "No news in this list":

Do you always get “no news in this list” (in German: “Keine News in dieser Ansicht”) although tt_news is correctly set up? Then maybe it’s because you have a LATEST plugin somewhere and the items shown there are not displayed in a LIST anymore. To avoid this, add

plugin.tt_news.excludeAlreadyDisplayedNews = 0

to your template (TypoScript setup).

There is a documentation bug that states it’s off by default, but it’s on…

[Typo3] Install a cronjob yourself without hoster assistance

Sun, 12 Feb 2012 00:08:40 +0100

If you host a Typo3 installation at a provider where you don’t have shell access, need a custom cron job for the Typo3 scheduler and the host’s configuration interface (Plesk, for instance) is somehow cumbersome (for instance, it may limit your cron job to a chrooted shell and PHP isn’t installed the chroot environment, so you can’t start PHP scripts from there), you can try to install a Typo3 Web shell extension. Then check for existing cron jobs:

crontab -l

If you see some environment variable definitions, remember them. There may be a SHELL= restricting you to a chroot shell – in this case you may choose to remove this environment variable in your custom crontab. Now write your custom crontab, put it into fileadmin and call

crontab fileadmin/your-cron

Check with

crontab -l

It should work because in professional hosting environments, there shold be only one user for Web, FTP and cron job and by default, every user can set up his own cron jobs in UNIX.

[Typo3] Using direct_mail with external SMTP server

Wed, 08 Feb 2012 00:22:00 +0100

There are several traps when using Typo3 4.5 and the direct_mail newsletter extension with an external SMTP server. Here’s how to set it up:

First of all, prepare the SMTP server (for instance, by allowing the IP of the Web server as relay host or setting up user authentication).

In the Typo3 Install tool, set the [MAIL] settings correctly (transport=smtp, transport_smtp_server, defaultMailFromAddress and defaultMailFromName). To see if it works, send a mail from within the Install tool and check its headers (the SMTP server should be mentioned in a Received line).

As a dependency for direct_mail SMTP usage, you need PEAR::Mail with Net::SMTP support. Mail.php must be in the include path. Install it on your server, or if you only have a Webspace, put the necessary files into a /pear directory and add this directory to the include_path. Check whether it works using a modified example script. Don’t change the path in the include line because direct_mail will include Mail.php directly, too.

Then set the SMTP server in the Typo3 extension manager’s direct_mail settings.

Always try out newsletters with separate test recipients because the single test mail in the last step of sending isn’t exactly the same how the other emails are sent. If you get an error at the last step (step 5), set [MAIL][substituteOldMailAPI]=0.

If you have debug messages in the mail body, turn off [FE][DEBUG] in the Install tool or localconf.php.

If you use the “Direct Mail Plain text” static template as well as “Direct Mail Content Boundaries” and your emails are screwed up, make sure that “Direct Mail Content Boundaries” are included before anything else.

To disable CSS and JavaScript output in the HTML version of mails, add this to your TypoScript setup:

config {
  removeDefaultJS = 1
  inlineStyle2TempFile = 0
}
plugin.tx_cssstyledcontent._CSS_DEFAULT_STYLE >

Avoid special characters like ’ and _ in the direct_mail “sender name” because there is a bug that such mails won’t be sent over PEAR::Mail. This bug should have been fixed since 2009, but it’s still present for me.

If you try to send an HTML / HTML+plain text version but you only receive plain text, make sure the recipient (tt_address) records have “Enable HTML mail” set.

Cisco WebVPN: "applet not initialized"

Wed, 01 Feb 2012 23:55:04 +0100

If you connect to Cisco WebVPN with port-forwarding and get the message “applet not initialized” although Java applets work in your browser, you may have to enable third-party cookies in your browser.

Converting DBF to CSV files

Wed, 21 Dec 2011 15:47:00 +0100

How to convert DBF to CSV files:

1) Install PERL, DBI, and DBD::XBase (comes as a package for many Linux distributions). 2) Use a script like this one:

#!/usr/bin/perl
use DBI;

my $dbh = DBI->connect("DBI:XBase:/directory/of/dbf/file") or die;
my $sth = $dbh->prepare("select * from YourDbfNameWithoutFileExtension") or die;
$sth->execute() or die $sth->errstr();

# adapt column names
print "Col1Name,Col2Name,Col3Name\n";

my @data;
while (my @data = $sth->fetchrow_array()) {
    print join(",",@data)."\n";
}

3) Call this script and redirect stdout to the CSV file.

Android: generate incoming SMS from within your app

Sun, 11 Dec 2011 23:27:45 +0100

(Tested with Android 2.3.3 SDK)

Sometimes, you need to simulate an incoming short message for your app. For instance, an encryption app could need to intercept incoming SMS using a BroadcastReceiver, then check if the SMS was encrypted and if so, decrypt it. Then the decrypted SMS and not the original one should be processed further (show up in SMS app etc.).

The usual way is to create a BroadcastReceiver, intercept the SMS_RECEIVED broadcast on the incoming message and abort the broadcast if necessary (if the message was encrypted or somehow else only for your app). Then you could generate a notifcation and put the SMS into the inbox:

Putting an SMS directy into the native inbox database

You will need the WRITE_SMS privilege.

public static final String SMS_EXTRA_NAME = "pdus";
public static final String SMS_URI = "content://sms";

public static final String ADDRESS = "address";
public static final String PERSON = "person";
public static final String DATE = "date";
public static final String READ = "read";
public static final String STATUS = "status";
public static final String TYPE = "type";
public static final String BODY = "body";
public static final String SEEN = "seen";

public static final int MESSAGE_TYPE_INBOX = 1;
public static final int MESSAGE_TYPE_SENT = 2;

public static final int MESSAGE_IS_NOT_READ = 0;
public static final int MESSAGE_IS_READ = 1;

public static final int MESSAGE_IS_NOT_SEEN = 0;
public static final int MESSAGE_IS_SEEN = 1;

private void putSmsToDatabase( ContentResolver contentResolver, SmsMessage sms )
{
    ContentValues values = new ContentValues();
    values.put( ADDRESS, sms.getOriginatingAddress() );
    values.put( DATE, sms.getTimestampMillis() );
    values.put( READ, MESSAGE_IS_NOT_READ );
    values.put( STATUS, sms.getStatus() );
    values.put( TYPE, MESSAGE_TYPE_INBOX );
    values.put( SEEN, MESSAGE_IS_NOT_SEEN );
    values.put( BODY, sms.getMessageBody().toString() );

    // Push row into the SMS table
    contentResolver.insert( Uri.parse( SMS_URI ), values );
}

Source: http://stackoverflow.com/q/5946262 (modified)

However, this method is not future-safe because the content://sms URI is not specified in the SDK and may be subject to change. Also, if the user uses another SMS app than the default one, the message might not appear in the inbox of this SMS app.

Doing it by generating a SMS_RECEIVED broadcast

So I thought about the possibility to generate a SMS_RECEIVED broadcast by the app. So the message would be processed as a “real” one, including all other filters and then go into the correct SMS app, including all notifications just as from a real one.

A Google research told me that it was not possible: http://stackoverflow.com/q/3819856. I tried myself and after setting the BROADCAST_SMS permission, I didn’t get a permission error but didn’t manage to get it working. Then I found this useful posting: http://d.hatena.ne.jp/thorikawa/20100930/p1 (use a translation service if you don’t speak Japanese – I don’t) and got it working (just call sendSms from your app):

You will need the BROADCAST_SMS privilege.

     private static void sendSms(Context context, String sender, String body) {
             byte [] pdu = null ;
             byte [] scBytes = PhoneNumberUtils
                             .networkPortionToCalledPartyBCD( "0000000000" );
             byte [] senderBytes = PhoneNumberUtils
                             .networkPortionToCalledPartyBCD(sender);
             int lsmcs = scBytes.length;
             byte [] dateBytes = new byte [ 7 ];
             Calendar calendar = new GregorianCalendar();
             dateBytes[ 0 ] = reverseByte(( byte ) (calendar.get(Calendar.YEAR)));
             dateBytes[ 1 ] = reverseByte(( byte ) (calendar.get(Calendar.MONTH) + 1 ));
             dateBytes[ 2 ] = reverseByte(( byte ) (calendar.get(Calendar.DAY_OF_MONTH)));
             dateBytes[ 3 ] = reverseByte(( byte ) (calendar.get(Calendar.HOUR_OF_DAY)));
             dateBytes[ 4 ] = reverseByte(( byte ) (calendar.get(Calendar.MINUTE)));
             dateBytes[ 5 ] = reverseByte(( byte ) (calendar.get(Calendar.SECOND)));
             dateBytes[ 6 ] = reverseByte(( byte ) ((calendar.get(Calendar.ZONE_OFFSET) + calendar
                             .get(Calendar.DST_OFFSET)) / ( 60 * 1000 * 15 )));
             try {
                     ByteArrayOutputStream bo = new ByteArrayOutputStream();
                     bo.write(lsmcs);
                     bo.write(scBytes);
                     bo.write( 0x04 );
                     bo.write(( byte ) sender.length());
                     bo.write(senderBytes);
                     bo.write( 0x00 );
                     bo.write( 0x00 );  // encoding: 0 for default 7bit
                     bo.write(dateBytes);
                     try {
                             byte[] bodybytes  = GsmAlphabet.stringToGsm7BitPacked(body);
                             bo.write(bodybytes);
                     } catch(Exception e) {}

                     pdu = bo.toByteArray();
             } catch (IOException e) {
             }

             Intent intent = new Intent();
             intent.setClassName( "com.android.mms" ,
                             "com.android.mms.transaction.SmsReceiverService" );
             intent.setAction( "android.provider.Telephony.SMS_RECEIVED" );
             intent.putExtra( "pdus" , new Object[] { pdu });
             context.startService(intent);
     }

     private static byte reverseByte( byte b) {
             return ( byte ) ((b & 0xF0 ) >> 4 | (b & 0x0F ) << 4 );
     }

Source: http://d.hatena.ne.jp/thorikawa/20100930/p1 (modified)

I have used GsmAlphabet from the Android source code (I changed all EncodingExceptions to generic Exceptions for testing purposes).

For a good reference of how to create a PDU and what the single fields mean, please have a look at http://www.dreamfabric.com/sms/.

How to restart god after deploying a Rails app with Capistrano

Wed, 09 Nov 2011 00:43:53 +0100

If you use the God gem to manage your Ruby on Rails application server (Unicorn, for example), you will have to restart the application server using God after deploying a new version.

I found a solution that polls the application’s tmp directory for restart.txt (like Phusion Passenger does) from within the god configuration and restarts the app if the file is touched. However, it didn’t work well for me because the ps syntax varies on different systems and the method is not very reliable (it triggered when I didn’t want it to, and vice versa).

So I thought about it and now I think that the best solution to restart an app via god is to do (surprise):

god restart YOUR-APP

And it should be done exactly if capistrano deploys, and not by polling.

To allow this command to be run by your deploying user, install sudo and make an entry into sudoers. It may look like this:

thedeployinguser ALL=(root) NOPASSWD: /usr/local/bin/god restart YourRailsServer

Then, all you need to do in config/deploy.rb is to make it look like this:

namespace :deploy do
  task :start do ; end
  task :stop do ; end
  task :restart, :roles => :app, :except => { :no_release => true } do
     run "ln -nfs #{shared_path}/sockets #{release_path}/tmp/sockets"
     sudo "god restart YourRailsServer"
  end
end

(I need the socket symlink for the Unicorn UNIX sockets.)

[FreeBSD] unicorn.god

Sun, 23 Oct 2011 18:41:00 +0200

I had a very strange problem concerning my Ruby on Rails installation, using FreeBSD, nginx, Unicorn and God.

When starting god from the command line as root, everything worked. However, as soon as I placed the god command into /etc/rc.local or a even service file in /usr/local/etc/rc.d, god didn’t start the Unicorn server but showed exit code 127 for “bundle exec unicorn -E production -c config/unicorn.rb”.

So I added a log directive to the god configuration and looked at the log file. It said:

env: ruby1.9: No such file or directory

Then I changed the god configuration to include /usr/local/bin in the PATH and now everything works. Here my configuration file:

RAILS_ROOT = '/srv/www/my-project-prod/current'

God.watch do |w|
    w.name = 'my-project'
    w.interval = 30.seconds

    w.uid = 'my-project'
    w.gid = 'my-project'

    w.pid_file = File.join(RAILS_ROOT, "tmp/pids/unicorn.pid")
    w.behavior(:clean_pid_file)

    w.dir = RAILS_ROOT
    w.env = { 'PATH' => '/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin' }

    w.start = "bundle exec unicorn -E production -c config/unicorn.rb -D"
    w.stop_signal = 'QUIT'
    w.restart = "kill -USR2 `cat #{w.pid_file}`"
    w.start_grace = w.restart_grace = 30

    w.start_if do |start|
        start.condition(:process_running) do |c|
            c.running = false
        end
    end
end

If somebody is interested, here is my god service file (/usr/local/etc/rc.d/god):

#!/bin/sh
#

# PROVIDE: god
# REQUIRE: DAEMON

. /etc/rc.subr

name="god"
rcvar=`set_rcvar`
command="/usr/local/bin/god"

start_cmd='${command} -c /usr/local/etc/god.conf'
status_cmd='${command} status'
stop_cmd='${command} terminate'

load_rc_config $name
run_rc_command "$1"

Where god_enable=”YES” is set in /etc/rc.conf, and /usr/local/etc/god.conf includes all my project god files.

hey, when u installed ts3 on freebsd, did u get the serveradmin password??

Thu, 29 Sep 2011 00:59:46 +0200

yes, it is in the ts3 log file

Ruby 1.9 invalid multibyte char (US-ASCII) when running bundler via Capistrano on FreeBSD

Sun, 11 Sep 2011 14:26:53 +0200

If your Rails app works fine after upgrading to Ruby 1.9 (including bundler install in development environment), but throws an error like this when you deploy it via Capistrano:

... capistrano runs bundler as deployment user ...
... invalid multibyte char (US-ASCII) ...

make sure that the environment variables are set correctly for this user, especially that the LANG variable sets UTF-8 as charset (for instance, LANG=en_US.UTF-8). Of course, you can just set it in the shell profile of the app user, or you can do it the FreeBSD way in login.conf.

On-the-fly RSA encryption for secure transmission of forms/resources over non-secure connections

Thu, 11 Aug 2011 23:02:37 +0200

On-the-fly RSA encryption for secure transmission of forms/resources over non-secure connections:

I created a new plug-in / gem called acts_as_securely_transferable. It provides on-the-fly RSA encryption for secure transmission of forms/resources over non-secure connections.

See README for more details.

Why pay for an SSL certificate when JavaScript does it, too?

Mon, 08 Aug 2011 19:05:19 +0200

Why pay for an SSL certificate when JavaScript does it, too?:

This is a very nice solution to encrypt password fields, credit card numbers etc. in a secure way without needing an expensive SSL certificate. Of course, the authenticity function is lost (i.e. the end user can’t verify who you are, but anybody can buy an SSL certificate too so it doesn’t make much difference). The important thing is the secure transmission of the secret details.

It makes use of this RSA implementation in JavaScript. Don’t store secret information in a CookieStore session (which is used in Rails by default now) - use, for instance, EncryptedCookieStore.

Why it doesn't make any sense to move invalid HTML code into JavaScript

Sat, 06 Aug 2011 00:15:56 +0200

When I added the Google +1 button to one of my Web sites, I noticed that the button code given by Google is not valid HTML 5. There is a HTML 5 version, but when used with i18n (for instance, in German) the JavaScript code is:

Now this isn’t valid HTML 5 because the script tag can have either a source reference (src) or some inline script content.

So I began to search the Web and found solutions like this:

$(function(){
  $('#gplus').append('');
});

Why doesn’t make this any sense?

Short answer: If you read a newspaper and don’t understand the text - would it be helpful to find instructions how you can write the doubtful text by yourself?

Long answer: Let’s first ask why you want your code to be valid HTML. Probably to get the greatest possible compatibility with current and future browsers and to support the development of the Web. Who do you write the HTML code for?

User agents: Current and future user agents should be able to understand and render your code. In the actual case of the +1 button, user agents may either understand the code and render it or they simply ignore it. There is no need to “hide” the HTML code for them and then insert it by JavaScript because they have to interpret the code inserted by JavaScript nevertheless.
Robots: Normally, they don’t use JavaScript so they would be the target group for “hiding HTML by JS”. However, robots are designed to get most information of the Web and can parse most code that is somehow understandable. In case of correctly opened and closed tags, there is no reason why a robot shouldn’t understand a page that contains an unknown tag. Also, the most important robot is Google and I doubt that Google will reject your page because you embedded the Google +1 button. Furthermore, the actual problem is related to the
For yourself: Do you really feel better when a HTML validator tells you that your code is valid, but you know that it inserts invalid code by JavaScript?

Of course, I have a strong preference for really valid Web code - but moving invalid code from HTML to JavaScript doesn’t make any sense for me.

Google: Please give me the -1 button

Tue, 02 Aug 2011 23:35:00 +0200

Where I can vote down (only for me! and maybe Google friends) all these crappy sites that are spamming the search results.

FreeBSD: finding out free/total memory

Wed, 27 Jul 2011 16:00:24 +0200

FreeBSD: finding out free/total memory:

As there is no “free” like in Linux by default, I found this script which gives a good overview about the current RAM usage on FreeBSD systems.

The “freecolor” port doesn’t work for me because it’s only available for i386 systems.

Rails security: mass assignments allow assocations to be changed by user

Fri, 22 Jul 2011 17:52:00 +0200

Inspired by http://railspikes.com/2008/9/22/is-your-rails-application-safe-from-mass-assignment

If you’re using associations, they can be set by mass-assignment by default. For instance:

class Property
  has_many :rooms
end

class Room
  belongs_to :property
end

Assuming this code for room creation:

@property = Property.find params[:id]
@room = @property.rooms.create params[:room]

This will work fine because even if params[:room] contains { :property_id => some_other_id }, the property of @room will be set to @property.

However, if you have an update with mass-assignment like this:

@property = Property.find params[:id]
@room = @property.rooms.find params[:room_id]
@room.update_attributes params[:room]

it is possible to change the belongs_to association by passing params[:room] => { :property_id => some_other_id }.

Of course, sometimes this is desired behaviour (think of a property user who can select which room they belong to). However, often this is a huge security hole.

The only solution I know is to use attr_accessible for all models. It’s also possible to force attr_accessible for all models by creating this initializer:

ActiveRecord::Base.send(:attr_accessible, nil)

simple_format, auto_link and :rel => 'nofollow'

Tue, 21 Jun 2011 15:38:00 +0200

Rails 3.0.6+

If you use auto_link to mark up links in your text, you can add the rel=”nofollow” attribute like this:

auto_link(@text, :html => { :rel => 'nofollow' })

However, if you also want to use simple_format:

simple_format(@text)

you can’t combine this by default:

simple_format(auto_link(@text, :html => { :rel => 'nofollow' }))

because then the rel=”nofollow” will be filtered by the sanitation initiated by simple_format. So, you can either tell it to don’t do sanitation or just add this into a config/initializer:

MyApp::Application.config.after_initialize do |config|
  ActionView::Base.sanitized_allowed_attributes = 'rel'
end

This should work too (but doesn’t for me all times):

MyApp::Application.config.action_view.sanitized_allowed_attributes = 'rel'

Then this should work:

simple_format(auto_link(@text, :html => { :rel => 'nofollow' }))

Installing TeamSpeak 3 server on FreeBSD

Sat, 14 May 2011 23:39:00 +0200

I have made a FreeBSD port for the TeamSpeak 3 server (audio/teamspeak3-server), for amd64 and i386 architectures.

To install the port:

cd /usr/ports/audio/teamspeak3-server
make
make install

or, if you have installed portupgrade:

portinstall teamspeak3-server

It adds a teamspeak system user automatically, installs TeamSpeak into the correct directories and adds an rc.d entry. To enable the service, add

teamspeak_enable="YES"

to your /etc/rc.d and it will be started when the system boots. Use

service teamspeak status
service teamspeak start
service teamspeak stop

to manage it at runtime.

If you have a license file (licensekey.dat), put it into /usr/local/etc/teamspeak. Logs (they comprise the admin token after the first start) are stored in /var/log/teamspeak, the files at /var/db/teamspeak.

On the TeamSpeak download page, they say that one is not allowed to link to these files directly without their written permission so a port would not be allowed. However, I mailed them regarding the port and they told me that I can announce it in the forums so I think this is a permission.

Xerox Phaser 6110N network password reset

Sun, 03 Apr 2011 18:00:00 +0200

After studying the manual, the service manual, various Internet sites, 3 calls to the Xerox hotline and a lot of help-requesting mails to Xerox partners (not a single one answered even when I offered money for a simple answer), I can now say that:

The Xerox Phaser 6110N network password (for the Web interface) cannot be reset. If you forget it, you can’t use the networking functionality anymore and you have downgraded it to a Phaser 6110.
The IP address can only be changed with a Windows tool - useless if you use Linux or Mac.

So: Don’t buy it.

Gigabyte "Easy Energy Saver" not so easy without Windows

Fri, 18 Feb 2011 19:34:00 +0100

Me: Regarding the "Easy Energy Saver" function: Is it a hardware feature that is always available or do I need the Windows EES application/Windows drivers to run? I only use Ubuntu Linux on my system and I'm interested in maximum energy saving.

Gigabyte: [...] It's hardware oriented designed but needed to be driven by software.
There is no software for Linux.
Officially we don't support any Linux distribution.

Me (thinking): "There is no software for Linux", so energy saving is only available if you buy and use Microsoft(R) Windows(R) all the time. With any other operating system, you have to waste electrical power and harm the environment, we're sorry.