Disable Asterisk call loop detection: SIP 482 check

Asterisk does not allow to loop sip calls. This is in fact the correct behavior as specified in the SIP RFC. However, in some cases this might be very well the behavior that someone wants.

What loop detection prevents is essentially that an Asterisk instance does not accept an incoming call that was triggered by himself.

This might happen, if the Asterisk:

1. Tries to call himself (I had a good reason to do this**)
2. Asterisk tries to make a SIP call to another SIP peer, but the call is routed back to the Asterisk.
   This might happen if Asterisk registers with two or more accounts at an external SIP provider and one of account tries to call the other.

How to disable loop detection

There is no default option provided to deactivate this feature (as this should only be done, if you know what you are doing).

Loop detection is done in channel/chan_sip.c in function handle_request_invite(...). On line 22325 (Asterisk 1.8) a check is done, if loop detection should be done. You can also search for "482 Loop Detected". If this if-statement is disabled (add 0 == 1 &&) or removed, no loop detection will be done.

Re-compile and install as I described here.

** I am using Asterisk for Transcoding twice in the same instance. Basically Asterisk fowards an incoming SIP-call to himself (setting SIP_CODEC_OUTBOUND to whatever I needed) and then relays the call to the callee:

Caller Client --(slin)-> Asterisk --(whatever) -> Asterisk --(slin)-> Callee Client

Patch a asterisk module without breaking package management on Ubuntu

I needed to modify one module (chan_sip) of Asterisk, but was to lazy to download and install Asterisk manually.

1. Install build-tools
   apt-get install build-essential
2. Get the source and build dependencies
   apt-get source asterisk
   apt-get build-dep asterisk
3. Patch or do whatever you wanted.
   ....
4. Re-compile Asterisk
   ./configure && make
5. Install the Asterisk module
   cp channels/chan_sip.so /usr/lib/asterisk/modules/

PS: Was done on Ubuntu 13.10.

Bringing your contacts to a Nokia 7110

I  just needed a new phone and decided to go for cool solution: Nokia 7110 (v4.84).
I got one including serial data cable + dock and the only thing missing was: how do I get my contacts on the phone without typing?

The answer was quite simply: export your address book as vCard and send it to the phone via gnokii on an Ubuntu.

1. Get your address book
   Export it somehow: I tried Evolution as well as GMail web-interface (both work).
2. Get a serial port
   Setting up the serial port on my Lenovo X60 was not straight forward, but now it works...
   Really depends on your hardware
3. Connect to phone
   Use minicom and check if the phone replies to AT commands
4. Configure gnokii
   [global]
   port = /dev/ttyS0
   model = 7110
   connection = serial
   use_locking = yes
   serial_baudrate = 9600
5. Use gnokii to read/write your address book
   gnokii --deletephonebook ME 1 200
   gnokii --writephonebook -o --vcard < addressbook.vcf

The Nokia 7110 has a bug: the vCard-parser is broken...

So, I needed to fix my addressbook: basically remove all not needed data (emails, addresses, birthdays etc.). I implemented a small python-script using vObject.
It reads from stdin and writes to stdout: cat addressbook.vcf | python script.py

#!/usr/bin/python
#encoding=UTF-8

import vobject
import sys
import inspect

#Remove entries except in list for one vcard.
def vcard3_remove_all_entries_except(vcard):
    allowedEntries = {'version', 'fn', 'tel', 'categories'}
    entries_dict = vcard.contents
    for key in entries_dict.keys():
        if key not in allowedEntries:
            del entries_dict[key]

importedVCards = vobject.readComponents("".join(sys.stdin.readlines()))
try:
    while (True):
        vcard = importedVCards.next()
        vcard3_remove_all_entries_except(vcard)

        vcard.add('n');
        try:
#            vcard.prettyPrint()
            print(vcard.serialize()),
        except Exception as e:
            print(e)
            
except StopIteration:
    None

Sync with style:
cat contacts.vcf | python import.py | gnokii --writephonebook -o --vcard

And a tutorial to clean the Nokia 7110 can be found here.

Transcode MKV (h264) with multiple audio streams to MP4 (AAC)

Transcoding MKV containers using avconv to MP4, so that the files can be processed with Adobe Premiere. The following command copies the video stream (should already be in h264)  and convert all audio streams to AAC. All audio streams are sampled to 6 channels (even if the source is only stereo).

avconv -i INPUT.mkv -c:v copy -map 0:0 -c:a aac -map 0:1 -map 0:2 -ab 448k -ac 6 OUTPUT.mp4

The example is for 2 audio streams; just add more -map commands as needed.

PJSIP for Ubuntu (PPA) with video support

I am happy to announce that PJSIP 2.1 is packed for Ubuntu 13.10 including Video support.

PPA here: https://launchpad.net/~dennis.guse/+archive/sip-tools

On Ubuntu do the following:
sudo add-apt-repository ppa:dennis.guse/sip-tools
sudo apt-get update sudo apt-get install libpjsip-samples python-pjsip

Samples reside then in /usr/lib/libpjsip-samples/

Features:

• Python 2.7 bindings
• SSL
• (new) Video support
• (new) vidgui sample application [is patched]

Next steps:

• Add x264
• Python bindings: Add video support
• Python bindings: Bump bindings to Python 3.X

openHKP [initial release]

OpenHKP release mail send to openPGP.js:

Hey,
first of all thanks for building openpgp.js!
Some weeks ago I started using openpgp.js and found it pretty convenient.
However, I missed one feature: the interaction with PGP-Keyserver.
Long story short: As the keyserver protocol (HKP) is basically HTTP, I created a brief implementation in Javascript:  https://gitorious.org/openhkp-js/openhkp-js
Furthermore, I setup a CORS-enabled proxy for some keyservers: http://g00se.indus.uberspace.de/
The implementation is based upon: http://tools.ietf.org/html/draft-shaw-openpgp-hkp-00
Best regards,
---
Dennis Guse

Asterisk: Accepted Patch

First contribution to Asterisk (implemented together with Frank Haase):

For video calls, we would like to set the codecs in the dialplan using
SIP_CODEC. However, if SIP_CODEC is set, all codecs except the ONE set are disallowed and thus either audio or video is available.

Attached is a patch for 11.4 that allows SIP_CODEC to contain a list of codecs , e.g. "gsm,h264".

https://issues.asterisk.org/jira/browse/ASTERISK-21976

is accepted: https://reviewboard.asterisk.org/r/2728/

Thanks to Matt Jordan and Rusty Newton (both Digium) for their reviews.

PJSIP library package for Ubuntu

I find it pretty annoying to compile PJSIP on all hosts that I use by hand.

It is quite error-prone and the procedure is quite time-consuming.

So, I created the scripts to setup an Ubuntu packet (at the moment only 13.04). It is available as PPA here: https://launchpad.net/~dennis.guse/+archive/sip-tools

On Ubuntu do the following:

sudo add-apt-repository ppa:dennisguse/sip-tools

sudo apt-get update

sudo apt-get install libpjsip-dev python-pjsip

Features:

* Python-bindings

* SSL

Missing Features:

* Video (some dependencies are missing in Ubuntu 13.04 and will come with 13.10).

Reverse proxy with CORS usin Apache2

For the implementation of a Javascript-Client, I needed a CORS enabled reverse-proxy.

Here is one using apache2 (Virtual Host):

LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so 
LoadModule headers_module /usr/lib/apache2/modules/mod_headers.so


 ProxyRequests Off

 ProxyPass / http://SERVER
 ProxyPassReverse / http://SERVER

 Header set Access-Control-Allow-Origin "*"
 Header set Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"

Here is one using apache2 (.htaccess): mod_rewrite and mod_headers must be loaded.

RewriteEngine  On
RewriteBase /
RewriteRule  (.*)  http://SERVER/$1  [P,L]

 Header set Access-Control-Allow-Origin "*"
 Header set Access-Control-Allow-Headers "Origin, X-Requested-With, Content-Type, Accept"

PS: Be aware of the security implications!

Pairing Bluetooth Keyboard on Windows and Linux using the same receiver

Found a solution on the web with authenticated connections here (the original description is far more detailed).

Short description (so that it is still available for me ;) ):

1. Pair with (each) Linux
2. Pair with Windows
3. Extract authentication key from Windows from windows registry (stored in Windows/System32/config/system) using libhivex-bin
   hivexml system | perl -ane 's/</\n</g; print' | grep -i 'key="YOUR BLUETOOTH ID WITHOUT : "' 
4. Decode the content of xml-entity with Base64
5. Replace Linux authentication key with the extracteted version Linux /var/lib/bluetooth/RECEIVERID/linkkeys
6. Restart Bluetooth service
   sudo service bluetooth restart

I can confirm that this works using Ubuntu 12.10 and Windows 8 with a Apple Wireless Keyboard.

Alternative solution without authentication (I WOULD NOT RECOMMEND THIS FOR A KEYBOARD) can be found here.

UPDATE: Still works with 13.04

HTML JS publish click-events to all childs

I have tables with one radio button per cell and its quite annoying to aim at the radio button completely...

<td onclick="for (var i in this.childNodes) 
if(typeof this.childNodes[i].click === 'function') 
    this.childNodes[i].click();">
  Just clickable text
  <input type="radio" />
</td>

CSipSimple - Getting started to hack it (tailor to my needs)

Just one or two things I need to remember about CSipSimple:

Infrastructure

• SipStack runs in background (communication via Parcable interface). It interacts with PJSIP.
  Important there is a watchdog timer (about 5s)!
• pjsua.java, pjsuaConstants is the SWIG generated interface to PJSIP
• Native functions never return boolean, check for pjSuaConstants.PJ_SUCCESS
• Be aware of string exchange with the native stack: PjSipService.pjStrToString(...)
• UAStateReceiver.java does the call creation and everything related to SipStack
• PjSipService.java is deeply integrated with UAStateReceiver.java (for example does recording...)

UI

• CustomDistribution.java defines some things for fast modifying the behavior
• Main screen is created in SipHome.java (incl. the slidable tabs)
  To remove tabs, just disable tab.add (except for the warning tab and the message tab [message disable in CustomDistribution.java]: may be triggered from outside)
• Notifications are handled in SipNotifications.java (but from Sip-Thread)
• DialerFragment.java is the dialer incl. call button and account switcher...
  Here the re-dial functionality is missing... (press the call button twice without number should redial previous number)
  
• InCallActivity.java :)
  Important call setup is done BEFORE launching the InCallActivity via SipManager.ACTION_SIP_CALL_UI from UAStateReceiver.launchCallHandler(...).
  There is even a activity start delay of 2000ms (UAStateReceiver.LAUNCH_TRIGGER_DELAY).
  I think it's not nice!!!! Because a user may hear sound of before the UI is ready, but there is might be a problem with race conditions...
• InCallControls.java is the ui element on the bottom of InCallActivity to control Bluetooth and loudspeaker etc. (Deeply integrated with InCallActivity: sadly NO MVC...)
• InCall.java is also related to Dialer

Bluetooth Remote Control for Windows Media Center

After years of using my Bluetooth mouse to control Windows Media Center (WMC) from my couch - I figured out that I needed a proper remote control (actually I visited a friend who used the IR remote of WMC). After endless time of searching I found the PS3 remote (Sony employs Bluetooth *yeah*) and it works with WMC using a small program that translates the game controller HID events to actual keystrokes.

It is called PS3BluMote (SRC via github) and written by Ben Barron.
As Ben's website was not reachable, I compiled from using VS2012 - took me about 2min.

And it works perfectly - I only need to figure out good keymapping (WMC key shortcuts are available here). In fact, I recommend to use for play/pause, skip and volume control the windows events (in PS3BlueMote: Media_) and not the WMC shortcuts - in this way also VLC and other players can be controlled.

Here are some additional startup options for WMC: I created a shortcut to go to the music section directly.

PS: I use it on Windows 8 Pro with WMC

(Rails3) ActiveRecords: how to implement a constructor and init own attributes

Today I encountered two nasty things during Rails development. My idea was to create a model class (< ActiveRecord:Base) that on creation create the current timestamp.
It is not possible to override initialize (at least it is doing nothing) as it is an ActiveRecord:Base.

I found the callback after_initialize defined by ActiveRecord, which is however not the correct way anymore. You should use it in macro style (source).

The next problem is that ActiveRecord overwrites the accessors and because ruby is not using the accessors by default if you are in the object, it must be called explicitely.

This works for me:

class A < ActiveRecord::Base
  attr_accessible :a
  after_initialize :init
  def init
    @a = "a"
    self[:a] = "b"
  end
end

Using GoogleDocs as Team editor for Latex-files

We wrote our last project paper using Latex which creates very nice looking documents, but is absolutely painful to work in teams. Even using a code management system like SVN/GIT won't make it really comfortable as you don't have comments and see changes of other editors live.

So, we put our Latex document in GoogleDocs (simply copied the content into a New GDocs Text Document) and shared it with our team. We also formatted the document by applying GDocs styles to our Latex headings, so the text looks structured and easier to work with.

At first we simply copied the GDocs content to a local file and compiled it. In fact, the copy-and-paste task is really annoying and so we developed a small bash script that automatically downloads the latest GDocs version and compiles it locally. So, in any case you need to setup your latex tools correctly, but don't waste your time in the compilation step.

You only need to enable sharing by URL and from this URL copy the document id, which you need to download the file.

Here is the script:

wget -O soups-article.download "https://docs.google.com/document/export?format=txt&id=PLACE_YOUR_ID_HERE" && \

iconv -c --from-code=UTF-8 --to-code=ISO-8859-1 soups-article.download | sed 's/end{document}*/end{document}/' > myFile.tex && \

pdflatex myFile && \

evince soups-article.pdf &

Asterisk and MessageSend

We are using Asterisk (running on FreeBSD 9.0 port stock version: 10.0.0-rc2) as SIP registrar/proxy. We are using the Android softphone cSipSimple, which also provides SIP SIMPLE messaging functionality. To enable this on Asterisk add to sip.conf: accept_outofcallmessage=yes and outofcall_message=message (the name of the context handler).


However, we had some trouble setting up Asterisk to really do it. In the message-ctx Asterisk provides the information of the current message as variables/function: MESSAGE(to), MESSAGE(from) and MESSAGE(body).

(First problem)
The first problem is that MESSAGE(to) contains the complete address, e.g. sip:PHONENUMBER@SERVERIP. If you try to use this address asterisk sends the message to itself and complains on receive that their is no routing for the incoming message. So, we need to remove the @IP part using the CUT command from the TO and the FROM. Now we can send a message from one phone to another: MESSAGESEND(CUTED_TO, CUTED_FROM).
(Second probleme)
However, we cannot reply as the recipient got as sender address something like sip:UNKNOWN@SERVERIP.
In short use MESSAGESEND(CUTED_TO, CUTED_FROM <CUTED_FROM>) or as example MESSAGESEND(sip:1, SOMEBODY <1>). The first part of the FROM is the name shown to the user and the second part the reply to phone number (without sip:).


Here is our working context for messaging of the extensions.conf:

[message]

exten => _X.,1,Set(TO=${CUT(MESSAGE(to),@,1)})

exten => _X.,n,Set(FROM=${CUT(MESSAGE(from),:,2)})

exten => _X.,n,Set(FROM_PHONE_NUMBER=${CUT(FROM,@,1)})
 exten => _X.,n,MessageSend(${TO}, ${FROM_PHONE_NUMBER}<${FROM_PHONE_NUMBER}>) 
 ;DEBUG Print the status of the MessageSend
exten => _X.,n,Verbose(0, ${TO} ${MESSAGE(from)} ${FROM} ${MESSAGE_SEND_STATUS}) 

Thanks to Nicolas.

Master-Thesis Template

Recently some of friends started to work on their theses and why reinvent a new Word Template every time?
So today, I finally found the time to strip down my master thesis word document and made it a word template.

I used and created it with Word 2010.

It has:
* Automatic lists for figures, tables (use stylesheets)
* Automatic table of contents (use stylesheets)
* Automatic page numbering (small latin for lists and ToC; arabic for text; big latin for the appendix)
* Automatic title of the current chapter on each text page.

You need:
* Something to write about
* Know how to use style sheets in word
* Somebody who reads your final text for mistakes
* and at last a covering page

Feel free to use it and don't waste your time creating yet another word thesis template.
And also feel free to modify and extend it.

The link to the template again.

Swissranger 4000 and OpenCV and Point Cloud Library

I am back on working with Depth Cameras and hand gesture recognition - now we are using a Swissranger SR4000 - a TOF camera - instead a Kinect. Here is demo code to get the camera data to OpenCV as well as Point Cloud Library.

/**
* Demo program for the SR4k that shows the output (DepthImage and PCL) and can export these as images.
* Use SPACE to take a picture and ESC for end.
 * @author Dennis Guse
 */
#include 
#include 
#include 
#include 

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/imgproc/imgproc_c.h"

//SR4k
#include "libMesaSR.h"
#include "definesSR.h"

#include "pcl/point_cloud.h"
#include "pcl/point_types.h"
#include "pcl/visualization/cloud_viewer.h"

using namespace cv;
using namespace std;
using namespace pcl;

#define SR_ROWS 176
#define SR_COLS 144
/**
 * Takes a picture with the SR4k and returns the depthimage as well as the point cloud!
 */
cv::Mat takePicture(SRCAM srCam, pcl::PointCloud::Ptr cloud) {
 SR_Acquire(srCam);
 cv::Mat depthImage(SR_ROWS, SR_COLS, SR_CV_PIXELTYPE, (unsigned short*)SR_GetImage(srCam, 0)); //0:DepthImage; 1:Amplitude; 2:ConfidenceMap

 float x [SR_ROWS * SR_COLS];
 float y [SR_ROWS * SR_COLS];
 float z [SR_ROWS * SR_COLS];
 SR_CoordTrfFlt(srCam, x, y, z, sizeof(float), sizeof(float), sizeof(float));

 for(int i=0; i<SR_ROWS * SR_COLS; i++) {
   point.x=x[i];
   point.y=y[i];
   point.z=z[i];
   point->points.push_back(point);
 }
 return depthImage;
}

int main(int argc, char **argv) {
 SRCAM srCam;
 SR_OpenETH(&srCam, SR_IP_ADDR); //Add error handling
 SR_SetMode(srCam, AM_COR_FIX_PTRN|AM_CONV_GRAY|AM_DENOISE_ANF|AM_CONF_MAP);

 pcl::visualization::CloudViewer viewer ("PCLViewer");
 while(true) {
  pcl::PointCloud::Ptr cloud(new pcl::PointCloud);
  cv::Mat depthImage = takePicture(srCam, cloud);
  cv::imshow("Depth", depthIamge);

  viewer.showCloud(cloud, "Cloud");

  int key = waitKey(1);
  if (key == KEY_ESC) break;
  if (key != -1) saveDepthImageAndCloud(depthImage, cloud);
 }
}

Lenovo X60 and Ericcson F5521gw

My old Lenovo X60 (wwan ready: antenna an sim slot available) uses the current state of the art UMTS (mini pci express) card from Ericcson. The card orignally belongs to an Lenovo W520 - FRU is 60Y3279.

!I am using the original BIOS (no zender!)

Basically:
* Download drivers for the card from Lenovo
* Extract drivers (the .exe complains that my X60 is not supported) using InstallShield tools
* Cover pin 20 on the card with scotch tape. (Drawback: card cannot be turned off)
* Plugin the card (I am using only the main antenna)

Hint:
* Even with pin 20 enabled (no scotch) my notebook booted without error 180X and windows showed it in the device manager - but cannot be turned on!
* Fn+F5 (Access Connection) does not show the card, but the connection can be configured.
* Access Connection can turn the card off (probably via AT commands)

PIN 20: Not my image; credits go to the unknown creator.

Any question: Leave a comment with your mail-addr.

PS: In comparison to WiFi tethering with my Palm Pre: the card is awesome fast! (using O2 max. 3.6 Mbit/s)

Update:
Similar guide for Dell and a WWAN

Extract drivers from encrypted install shield archives

Image of PIN 20

PS: If you need something, just send me an email.

Matlab: Progressbar for arrayfun

If you use arrayfun on large arrays with really slow functions it is annoying to wait without feedback how long it can take - so a progressbar (in matlab waitbar) would be nice. One problem is that arrayfun provides no information about the status - there are no callback handlers.
What could be done instead.
Simply write function for arrayfun as inner function of the function which calls arrayfun. In the calling function
you define two variables, one for the number of items and one for the current solved items (starts with 0 and gets updated). In the inner function the work is done and the counter increment, which is visible in the inner function, because it is an inner function. At last update the waitbar.
Here is how it can look:

function doArrayFunWithProgressBar()
  data = magic(35);
  WAITBAR = waitbar(0, 'Work in progress');
  PROGRESS_COUNTER = 0;
  PROGRESS_MAX = sum(size(data));

  data = arrayfun(@(item)innerFun(item), data);

  close(WAITBAR);

  function result= innerFun(item) 
    %TODO Do the work!
    pause(1);

    PROGRESS_COUNTER = PROGRESS_COUNTER + 1;
    waitbar(PROGRESS_COUNTER / PROGRESS_MAX);
  end
end

Matlab: if-statement in anonymous functions

UPDATE: The performance penalty is massive - MATLAB fails to optimize the code if anonymous functions or function handles are used. I got factor 20... But it works ;)

Today, I needed to write a function in Matlab as accessor to a matrix. I have a matrix which contain mulitple datastreams - one per row. The columns dimension is the time.

Because I don't want to change the functions, which evaluate only a subset of provided streams, and I don't want to copy and modify the matrix for every combination, I started to think about an accessor function.

Furthermore the function should reveal the dimensions of the (not existing) data matrix.

I ended up with anonymous functions, because these allow to access variables outside the anonymous function definition like:

a=2

b=2

fun=@(x)(a*x+b)

The advantage is that fun can use a and b, but the caller doesn't necessarily know that they exist and are used by fun.

With this knowledge the implementation of providing the correct data was straight forward. The next problem was, that in anonymous function you can't use the if-statement. That's pretty messy.

I found http://www.mathworks.com/matlabcentral/newsreader/view_thread/147044 a solution and adapted it.

%%Artificial if for use in anonymous functions
%TRUE and FALSE are function handles.
function RESULT = iff(CONDITION,TRUE,FALSE)
  if CONDITION
    RESULT = TRUE();
  else
    RESULT = FALSE();
  end
end

The function that creates the function handles of the accessor function:

function HANDLE = recordHandle(COLUMN, ROWS)
  HANDLE = @(row, column) (...
    iff(nargin == 2, @()COLUMN(ROWS(row), column), @()[size(COLUMN, 2), size(ROWS, 1)])...
  );
end

Apple Wireless Keyboard on Windows [AppleKeyboardInstaller.exe]

At the weekend I found a solution to a long lasting problem of mine. I have a Apple Wireless Keyboard, because two years ago it was the best Bluetooth keyboard on the market (Currently I can't say, didn't checked again). So, I use the keyboard every day mostly for writing text ;), but the device doens't posess a DEL, POS1 and END-Key and thus it is hard to navigate during typing.

The solution is install the keyboard driver provided by Apple in the Bootcamp package. The driver I use is from a OS X Leopard installation disc.
Or download it here: http://www.happytocode.com/post/Apple-Aluminium-keyboard-Boot-Camp-20-Windows-drivers.aspx

PS: After I found the solution myself, I had the correct google search term and solutions older than my keyboard. But it works.

UPDATE:
The keyboard driver sometimes leads to Bluescreens, so I am back using the standard windows drivers without the additional keyboard shortcuts.

Credit card: Visualize your money

Some time ago I had a discussion about (dis-)advantages of physical against non-physical stuff. She argued that non-physical is mostly too abstract for the user and thus not that easy to use in daily life.
We argued about "real" money as physical and credit cards as non-physical example. The advantages of real money are that you see what you have and you can only spend that. The advantages of the credit card are that the payment is easier and payment via internet is possible.

The main disadvantage is that you as user can't see how much money you spend or you have available. The card looks always the same.

Assume that materials are available that could either shrink or change it's color. Using this the credit card will get smaller if you pay with it and slowly reach your limit. Or the card could get orange if are in reaching line of credit and red if your are in.
Thus you as user would "feel" how much money is available and it is much easier to cope with the abstraction from real money.

PS: Only an experiment in mind.

Windows shell (cmd) and loops

FOR %t in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
DO FOR %g IN (1,2,3,4,5,6,7,8,9,10) DO ECHO %t %g

Hopefully I have never to use the windows shell again!

JSF 2.0: Mojarra and multipart/form-data (File Upload) [Glassfish]

It is a quite a mess that Mojarra doesn't support h:forms that use enctyp multipart/form-data, because you can't access possible available parts in the JSF Controllers.

The following wrapper extends a HttpServletRequest so that getParameter also uses available data in HttpServletRequest.getParts().

You can than access the HttpServletRequest via
FacesContext.getCurrentInstance().getExternalContext() and use getParts own your own.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.HashMap;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import javax.servlet.http.Part;

/**
 * The mojarra implementation does not parse POST request that are multipart encoded, 
 * because the parameters are not accessable via getParameter().
 *
 * This class extends the HttpServletRequest to provide access to the parameters
 * which are encoded accessable via getParts.
 *
 * All parts are made visible that have contentType == null && size < 300.
 *
 * If the request is not multipart encoded, the wrapper doesn't modify the behavior of the original HttpServletRequest.
 * @author dennis
 */
public class MultipartHTTPServletRequest extends HttpServletRequestWrapper {

    protected Map parameterParts = new HashMap();

    public MultipartHTTPServletRequest(HttpServletRequest request) {
        super(request);

        if (getContentType() == null) {
            return;
        }
        if (!getContentType().toLowerCase().startsWith("multipart/")) {
            return;
        }
        try {
            for (Part i : getParts()) {
                if (i.getContentType() == null && i.getSize() < 300) {
                    parameterParts.put(i.getName(), getData(i.getInputStream()));
                }
            }
        } catch (IOException ex) {
            Logger.getLogger(MultipartHTTPServletRequest.class.getName()).log(Level.SEVERE, null, ex);
        } catch (ServletException ex) {
            Logger.getLogger(MultipartHTTPServletRequest.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    private static String getData(InputStream input) throws IOException {
        String data = "";
        String line = "";
        BufferedReader reader = new BufferedReader(new InputStreamReader(input));
        while ((line = reader.readLine()) != null) {
            data += line;
        }
        return data;
    }

    @Override
    public String getParameter(String name) {
        String result = super.getParameter(name);
        if (result == null) {
            return parameterParts.get(name);
        }
        return result;
    }
}

Java Servlet (3.0): How can I access multipart encoded content provided by a http post request?

At the last friday i stood right before a tricky problem: How can I save a image provided via HTTP post into a database?
I had three problems:
1. How to upload a file via HTML?
2. How to access the data at the server side?
3. How to put in the database using JPA?

The first one was easy, just create a html form add a input field (type='file') and a submit button.

The second one cost me one day. And it was really simple: Just place the @MultipartConfig annotation at the class definition of the servlet and use HTTPRequest.getPart[s]() methods to access the data as an inputstream.

The last part was straight forward: use a InputStreamReader to copy the data into a byte[] and add @Lob byte[] field to the entity class.

Because I use MySQL it was necessary to change the columnt type from TEXT to MEDIUMBLOB.

Java: CDI & ConversationScope

Yesterday I recreated my web app project with maven. Now CDI is available and @ConversationScoped is really nice and makes the development a lot easier.
I used Glassfish v3.

... Post my pom.xml tomorrow again.

Goodbye 85.214.89.182 (g00se.org)

It's the last day of my good old vServer for g00se.org.
Tonight they will come and cut him off.

To keep him in memory old.g00se.org points to his former home adress.

Windows 7: Share your internet uplink via wireless ad-hoc network (ICS)

Scenario
You have one windows 7 device with a direct internet connection like UMTS and you won't to share the connection via an ad-hoc wireless network with another device. In my case it is a XDA neo.
Everything happens in the Network and Sharing center.

1. Properties section of the network device with the uplink:
   under sharing active ICS and re-connect the device to internet
   
2. Create a new ad-hoc network (use encryption!)
   
3. Join with another device the network
4. Set the wireless ad-hoc network location to "Home"
5. Disconnect from the ad-hoc network and rejoin (device with uplink)

The internet sharing should now work.

Ubuntu-vm-builder default login

Today I created a ubuntu virtual machines with ubuntu-vm-builder. When I tried to login nothing worked. I searched the web and found only the parameter --username, --password and --name. But I can't recreate two VM using an UMTS uplink :(.

I booted one machine (ubuntu jaunty VM) and used the password reset procedure to get a running root shell. In the passwd I found a user ubuntu. I rebooted and used ubuntu:ubuntu as credentials.
Default username: ubuntu
Default password: ubuntu

Thanks to NO documentation and man pages for ubuntu-vm-builder... Addon:
The packet is named python-vm-builder. See also https://help.ubuntu.com/community/JeOSVMBuilder.

CSS cheat sheet

Found by SOW: CSS cheat sheet.

SQL cheat sheet

Found by SOW: SQL cheat sheet.

HTML cheat sheet

Found by SOW: HTML cheat sheet.

PHP cheat sheet

Found by SOW: PHP cheat sheet.

Power consumption of the internet

Nice article....

SIP client for Windows (using ekiga SIP service)

After long time of crawling the web, I have found a solution to use the ekiga.net SIP service from Windows. Many of my friends use skype, but it is an annoying part of software. So I started using ekiga and ekiga.net as provider.

Microsoft itselfs provides a SIP client (netmeeting). The developer of Ekiga provides a nice howto.

Getting DBUS Messages from Networkmanager (Python)

I had some spare time and so I started playing with DBUS. I was annoyed that after NetworkManager established a connection I always start the same programs like ekiga, pidgin, firefox and evolution. So I wrote a small python program that start software if the NetworkManager singals an open connection via DBUS. The NetworkManager DBUS-API is available. The hardest part was to find the docs.

#!/usr/bin/python
from dbus.mainloop.glib import DBusGMainLoop
import gobject
import dbus
import subprocess

def signal_deviceNowActive(data=None):
  if data is not None and data[dbus.String("State")] == 2 :
    subprocess.Popen("ps -C pidgin    || pidgin &", shell=True)
    subprocess.Popen("ps -C ekiga     || ekiga &", shell=True)
    subprocess.Popen("ps -C evolution || evolution &", shell=True)

print "init loop"

DBusGMainLoop(set_as_default=True)

loop = gobject.MainLoop();

print "init dbus"

dbus.SystemBus().add_signal_receiver(signal_deviceNowActive, signal_name=None, dbus_interface="org.freedesktop.NetworkManager.Connection.Active")

print "start loop"

loop.run() //UPDATE: thanks for your comment

Cort Dougan: Good Programmers are Not Lazy

Made me knowingly  smile...

Book: Domain-Driven Design (Eric Evans)

In the last summer ('08) I had the time to read that book. A friend of mine (thanks to Thorben) suggested that the book is really worth a try. The cover says: "Tackling complexity in the Heart of Software". And that is a really summary.

Evans describes with short examples situations in software development projects which can lead to critical problems later. He uses the short stories to analyze the situation, make the problem and cause clear. Based upon that he describes a pattern (a style to develop software; mostly organizational stuff) to get rid of it.

The first one he suggests is a "ubiquitous language". In short he says that all people connected to the project should use the same language, so that everyone can talk to everyone about the domain. That is not about technical stuff or infrastructure. It is mere that all know all what the software should / will be used to.

It's a quite good book with well choosen practical examples and it helped to understand that real development differs from educational stuff in a sense of organization, time and size. It would have been helpful, if I read it earlier.

Lecture: Open Wins – the Future of Software & Technology (Scott McNealy, SUN)

Scott McNealy the chairmen and a co-founder of SUN visited the TU Berlin on 06-11-08. He visited the university to give a short talk about open source software and why it became and will be  a pushing element for technical innovations. Because he is with SUN the content of the talk was mostly about how  SUN was, is and will be involved in the development of open source.
The essance of the talk was:
    Open Source is good (not only code, also products [specifications])
        reducing the dependency between the developer and the user
        programmers improve their code style (someone else can read your stuff)
    Strategy of SUN
        He anwsered the question why SUN is doing open-source

Interesting points:
    Java
        happens transparently (ref: mobile phone and blue-ray players)
        is running on 6^9 devices (hope I got it right)
    Oracle Enterprise Edition costs 400.000 $ per core(!)
    
Cool quotes:
    "Free is the new black!"
    "Return on no-investment."
    "Steal software legally"
    "We don't want you to do .NET! Watch your hands!"

Provided links during the talk:
    Netbeans.tv: the community site of Netbeans.
    Solaris Express Developer Edition (SXDE) A developer edition of solaris with many tools

All in all I can say that Scott is a really good entertainer. It was a really nice talk!

Thanks to Prof. Brandenburg for the information.

JTree on an JScrollPane and resizing

Early on the morning I read my early morning bug reports ;).

It was about an JTree on a JScrollPane. The model of the will be modified during runtime and fires events if something is changes. So far everything looks pretty straight forward and it works.

The problem occured if the tree gots larger than the parent scrollpane, so I believed the scrollpane starts to activate the scrollbars and everything stays cool... But the tree never changed it's size. It simple stayed in the old size and the scrollpane didn't start scrolling. And so some items of the tree were invisible.

TreeModel model = new DefaultTreeModel();
JTree tree = new JTree(model);
tree.setPreferredSize(new Dimension(100, 100));
JScrollPane pane = new JScrollPane(tree);

The model fires an TreeModelEvent (DefaultTreeModel.nodeStructureChanged) and the tree get's an JTree.treeDidChange() so it can recompute it's size.

This doesn't work out. It was necessary to set the preferred size of the tree to null before excecuting treeDidChange...

Why? I honestly don't know...

VOIP for GNOME Desktop: Ekiga

Yesterday I got some time to start playing with SIP. I used Ekiga a SIP client for the gnome-desktop. After installation I got an account on ekiga.net and everything went fine. The provided configuration wizard is really cool stuff. The hole action consumed 10 minutes and afterwards I spent 1 hour to get the sound devices working. And it did, but it was really annoying. The only problem now is that my built-in (Thinkpad X60) microphone produces a feedback if I use the internal speakers. The only solution is to use headphones and don't type or click during a call.

At last: sip:500@ekiga.net is a echo service, so you can test your configuration. Next week I will try to get my bluetooth headset running.

JAX-WS: ORA-31011

Today, I tried to do something useful for my bachelor thesis. I tried to query a Oracle 11G DBMS via a SOAP-based Webservice. Using the instruction from Andrea and Oracle I got the service up and running. The Webservice was reachable under http://localhost:8080/orawsv and presented it's wsdl via http://localhost:8080/orawsv?wsdl.

Now the trouble started:
The URL from the Oracle HTTP-Server is secured via HTTP-Authentification. Ok so I downloaded the WSDL and created the stubs from a local file with the JDK's wsimport. Now I needed to tell the Webservice Client Provider to authenticate if necessary:

ORAWSVPortType port = new ORAWSVService().getORAWSVPort();
Map<String, Object> requestCtx = ((BindingProvider) port).getRequestContext();
requestCtx.put(BindingProvider.USERNAME_PROPERTY, "user");
requestCtx.put(BindingProvider.PASSWORD_PROPERTY, "password");

The first test ended with a desaster:

Exception in thread "main" java.lang.IllegalArgumentException: faultCode argument for createFault was passed NULL
at com.sun.xml.messaging.saaj.soap.ver1_1.SOAPFactory1_1Impl.createFault(SOAPFactory1_1Impl.java:56)
at com.sun.xml.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178)
at com.sun.xml.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:108)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:254)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:224)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:117)
at $Proxy32.xmlFromQuery(Unknown Source)
at productcatalogws.Main.main(Main.java:49)
Java Result: 1

I couldn't make anything useful out of these messages. The only thing I found was a dead end: bug_id=6587659.

So I started debugging:

First view the SOAPMessages:
I used the cool charles proxy.
Configuration for JAVA:

System.getProperties().put("proxySet", "true");
System.getProperties().put("proxyHost", "localhost");
System.getProperties().put("proxyPort", "8888");

After viewing the messages without noticing anything of interesst except:
ORA-31011: XML parsing error, but without any reference to the Webservice.

I found a cool tool to use webservices: soapUI (you can do everything I needed using it!!) and queried the Oracle Webservice by hand. And it worked!

The problem was that the default JAX-WS Provider does send:
Content-Type: text/xml;charset="utf-8"

And the Oracle HTTP Server expects:
Content-Type: text/xml;charset=UTF-8

An example SOAPMessage (including the header):

Authorization: Basic XXXXX
Host: localhost:8080
Content-Length: 314
SOAPAction: "http://localhost:8080/orawsv"
User-Agent: Jakarta Commons-HttpClient/3.0.1
Content-Type: text/xml;charset=UTF-8

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:oraw="http://xmlns.oracle.com/orawsv">
<soapenv:Header/>
<soapenv:Body>
<oraw:query>
<oraw:query_text type="SQL">SELECT * FROM PRODUCT</oraw:query_text>
</oraw:query>
</soapenv:Body>
</soapenv:Envelope>

The questions are:
Does JAX-WS something wrong during the request or is the DB Webservice the bad guy?
And why doesn't JAX-WS handle the SOAPfault correctly?

Used software:

javac --version:

java version "1.6.0_04"
Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode)

Oracle 11G DBMS:

Oracle Database 11g Release 1 for 32-bit Windows.

OpenSSH using Kerberos via GSSAPI

I missed to activate a small and tiny feature during the update to Debian Etch: OpenSSH with GSSAPI support. What does sat mean?

1. g00se.org uses Kerberos for authentification.
   
2. Kerberos offers Single-Sign-On.

I missed to upload my subversion working copies over ssh without typping my password everytime. So I installed ssh-krb5 to add the cool behavior.

Cyrus and Exim4 authentification using Kerberos via GSSAPI

Today I used my spare time to let the SMTP and the IMAP server of g00se.org using the GSSAPI for authentification.
The necessary cyrus-sasl libaries were already installed. So I really don't know which are exactly required. I suppose the cyrus-sasl gssapi libary should meet all requirements. I needed to install the exi4-daemon-heavy instead of the light one. The the heavy one does support authentification using the cyrus-sasl libary.
I created the principals imap/g00se.org and smtp/g00se.org and put them into the default keytab.
And modified the configuration files of both services to let them propose GSSAPI as alternate authentification mechanism:
(cyrus): imapd.conf:
sasl_mech_list: PLAIN GSSAPI
and
(exim4): [/etc/exim4/conf.d/auth/01_exim4-config_gssapi]
gssapi_server:
driver = cyrus_sasl
public_name = GSSAPI
server_mech = gssapi
server_hostname = g00se.org
#server_realm = G00SE.ORG
server_set_id = $auth1
.ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
.endif
Thanks to Sean for a short and easy description.
PS: Exim4 does use the splitted configuration file option of Debian. So you can put the lines anywhere into the authentification section.

Apache with Kerberos authentification

Special thanks to the guys who invented mod_auth_kerb. I removed the PAM authentification modules, which I only used as wrapper to get Kerberos auth through PAM and replaced it with mod_auth_kerb.
Here is the small configuration:

Krb5Keytab /etc/apache2/krb5.keytab
KrbAuthRealms G00SE.ORG
KrbServiceName HTTP
<Directory /x>
AuthType Kerberos
Require valid-user
</Directory>
That's all! Cool.

The Firefox bundled into my OpenSUSE 10.3 does already contain all necessary configurations. I only needed to add g00se.org to network.negotiate-auth.trusted-uris in about:config. So he does accept the offer to do GSSAPI authentification  with these URIS. And that's pretty cool. At least I need to figure a way to get my M$ system use such cool stuff.

Exim4 and Saslauthd [service=]

Hello! Since I upgraded g00se.org from Debian Sarge to Etch the authentification mechanism of my exim mail server doesn't run properly. I used as I descriped early the saslauthd to authentificate against the pam. Everything went fine. I believed, but a few weeks later (I used all the time my webinterface to send mails.) the exim couldn't use the same credentials as the imap server. The saslauthd always claimed that my credentials are false and so exim awnsered with a SMTP Error 535: Authentification failure. Tonight I managed to look into the problem and it's source. I checked the saslauthd using testsaslauthd. If I used: testsaslauthd -s smtp -u XXX - p XXX everything went fine and the saslauthd replied with credentials ok. But if I issued testsaslauthd -s "" -u XXX - p XXX I got a pam authentification error. I tried the same using exim and same behavior appeared, exim doesn't set the name of the service and so all authentification will fail. The problem is that the saslauthd will try to authenticate against a pam configuration which is not available. Note: I have in /etc/pam.d/ a file called smtp which defines the pam behavior for my smtp service ;). The messages (/var/log/auth.log): Dec 3 00:55:50 h1206589 saslauthd[22244]: do_auth : auth failure: [user=XXX] [service=] [realm=] [mech=pam] [reason=PAM auth error] Dec 3 01:18:45 h1206589 saslauthd[22247]: do_auth : auth success: [user=XXX] [service=imap] [realm=] [mech=pam] As you can see the name of the service is in the first log empty. I found a solution you can tell the exim how to call the saslauthd (Snip of the authentification part of my exim service): plain_saslauthd_server: driver = plaintext public_name = PLAIN server_condition = ${if saslauthd{{$auth2}{$auth3}}{1}{0}} server_set_id = $auth2 server_prompts = : .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}} .endif login_saslauthd_server: driver = plaintext public_name = LOGIN server_prompts = "Username:: : Password::" # don't send system passwords over unencrypted connections server_condition = ${if saslauthd{{$auth1}{$auth2}}{1}{0}} server_set_id = $auth1 .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}} .endif Now tell exim that he have to use a service for saslauthd change: server_condition = ${if saslauthd{{$auth2}{$auth3}}{1}{0}} into server_condition = ${if saslauthd{{$auth2}{$auth3}{smtp}}{1}{0}}. smtp have to be the name of the pam configuration file. That's all. Everythings work as expected.

JOptionPane and the Focus

I started to write a login screen (NEED: two input field [username, password]and two buttons [ok, cancel]). Since a login is a blocking panel I created a JPanel and put that on a JOptionPane.showDialog(...). Everything looked quite fine, but the default focus was set on the OK-Button. So I tried to requestDefaultFocus, but nothing worked. Mark provides a running solution. And it works!

WebSVN vs. ViewCVS

Tonight I removed ViewCVS and started using WebSVN. It's nice simple and in PHP :D... Take a look!

Java Base64 encoding (sun.misc.Base64Encoder)

Today I would like to say some words about BASE64 encoding and JAVA. Let me introduce a cool class to encode a string: sun.misc.Base64Encoder
This class seams to work quite all right for some weeks. So, today we transfered our JEE server from a linux to windows (not my idea). Till now I have assumed that the mythos of the JAVA plattform independence is not only a myth. During the check of the application (the deployed JEE project) it showed up error messages that the base64 coded strings doesn't match.
On the first look everything seemed to be fine. The second showed up that the strings on the windows machine were one (!) character longer. Two hours later we found the problem: Does RFC 3548 say something about line feeds and carriage return?
So why does the base64 coded strings contain some?
The anwser is: Because the encode method of the Base64Enconder split the string after some characters (I suppose at char 76 / 77, but I'm not quite sure). So if you switch the operation system and the new system uses another encoding for the line break, your old base64 encoded data is worthless.
To solve this problem I used the java mail api (cause JEE server needs to implement these):

StringOutputStream output = new StringOutputStream();
OutputStream encoding = MimeUtility.encode(output, "base64");
encoding.write("Hello World");
result = output.toString();

import java.io.IOException; import java.io.OutputStream;
/** * * @author Dennis Guse * */
public class StringOutputStream extends OutputStream {
  private StringBuffer data = new StringBuffer();
  public StringOutputStream() {}
  public String toString() {
    return data.toString();
  }
  public void write(int b) throws IOException { data.append((char)b); } }

So a half day of work for nothing....
PS: Feel free to use this stuff.

Authentification Server via Kerberos (Heimdal) and LDAP backend

Today morning I configured heimdal KDC and as storage the openldap slapd. Slapd stores the user information and holds the kerberos authentification information. The public URI is ldaps://g00se.org. Kerberos realm: G00SE.ORG. KDC: g00se.org The openldap server uses TLS and authenfication and authorisation with the SASL GSSAPI (package (debian): libsasl2-modules-gssapi-heimdal). First I installed slapd (slapd and the above package). Added the krb5-kdc.schema. Configurated SASL support and added authorisation rules. With slapadd -f "backup.ldif" I installed my backup (without internal kerberos accounts). Modified /etc/default/slapd to let the slapd listen on ldaps:// and ldapi://. Reloaded the configuration. Second I installed the heimdal-kdc. Configured the realm. Configured database backend to use the ldapi:// socket. Reloaded. Init the realm and create necessary kerberos host accounts (with random keys): kadmin -l > init G00SE.ORG > add -r host/g00se.org > ext_keytab host/g00se.org > add -r ldap/g00se.org > ext_keytab ldap/g00se.org Reloaded everything and the authenfication server was up and running! Have a nice one!!

Authenfication client via Kerberos (Heimdal) and LDAP backend

To use the provided authenfication mechanism from g00se.org on g00se.org :D I installed libpam-heimdal and configured nss. Kerberos configuration: cat /etc/krb5.conf [libdefaults] default_realm = G00SE.ORG default_keytab_name = /etc/krb5.keytab ticket_lifetime = 28800 default_etypes = des3-hmac-sha1 des-cbc-crc des-cbc-md5 default_etypes_des = des3-hmac-sha1 des-cbc-crc des-cbc-md5 [realms] G00SE.ORG = { kdc = g00se.org admin_server = g00se.org } [domain_realm] g00se.org = G00SE.ORG .g00se.org = G00SE.ORG The pam configuration to use the heimdal KDC: cat /etc/pam.d/kerberos #@include common-auth #@include common-account auth required pam_krb5.so account required pam_krb5.so Configuration of the nss-ldap plugin: cat /etc/libnss-ldap.conf uri ldaps://g00se.org/ ldap_version 3 base dc=g00se,dc=org scope sub pam_filter objectclass=account pam_login_attribute uid pam_min_uid 1000 pam_max_uid 2000 nss_base_passwd ou=People,dc=g00se,dc=org?one nss_base_group ou=group,dc=g00se,dc=org?oneh Let nss know that there is a second source which provide authorisation data. cat /etc/nsswitch.conf passwd: compat ldap group: compat ldap That's all! As root you can now check if everything runs: getent passwd and you will see all your local accounts and the provided central ones! I hope everything is cool!!!