No milk in my coffee

How were we successful before OSGI?

2011-08-05T02:20:00.000-07:00

According to some people discourse it would be a fair question, seems that modularity is impossible before OSGI. Wonder what we have been doing for the last 10 years...

Mind you, it was impossible to be successful without EJBs. So after sorting our threading issues with a well thought stack, now we have experts sorting our class loader and so far they are doing the same great job...

Spring ws Saaj vs Axiom

2011-08-02T08:14:00.000-07:00

Saaj create a lot of contention when manipulating the message - it seems to reload the an file from the class loader every time -, Axiom is much faster under stress.

to change the message factory create a new one in the application context under the name messageFactory

 

<bean name="messageFactory" class="org.springframework.ws.soap.axiom.AxiomSoapMessageFactory">
</bean>

Introducing Unit Testing in Legacy code base

2009-02-26T04:37:00.001-08:00

In big companies, developing a green field project is a rare privilege
... most of the time you will have to develop on a existing code base of
versatile quality - Not due to the skill of the developer more likely to
the history of the project, lack of code review and poor quality focus,
short term vs. long term risk -.

Anyway if you are not lucky you are facing a big spaghetti monster
calling other pasta entities in ways ranging from CORBA, RV, rmi, EJB,
ftp messaging ... Chances is that the code can't even run on your dev
machine, making development laborious and non productive.

There are 2 ways to deal with this problem

1 Status quo, self explanatory
2 Introducing unit test, mocking the internal and external monsters

1 : simple solution, not very rewarding but if the cost of solution 2
overcome the benefit that's the way to go
2 : long term benefit in development and quality. But costly

Though with a bit of method there are way to make 2 less costly, all you
will need is :

- A good refactoring dev environment, eclipse or idea
- Junit, XmlUnit
- XStream an xml serializer/desiralizer
- an UAT/Dev environment that runs.

1 - identify the part the function you need to test

you won't be able to test all the code, it is important to test a small
chain of the code lest that the mocking will be too complex.

It's likely that you can identify the function that is the entry and
exit point

Object f1(Object... args)

It is that function that we will try to unit test.

2 - record samples

your system is probably generating high complex object, based on high
complex entry. Trying to reproduce the graph manually is laborious and
sure to be incomplete. That's what we need to used XStream to record the
args and the output of the function in a uat environment, getting that
way real life object.



// the new method that will delegate to the old
// and dump in and out
Object f1(Object... args)  {
  dumpIn(args);

  Object o  = _f1(args);

  dumpOut(o);
}

// was the real
Object _f1(Object args) {
       ...
}

3 - integrate your sample in your unit test

Now that you have sample with your input parameters and your expected
result you can start writing your unit test.

4 - mock the external call

It is likely that once you integrate the sample in the unit test, the
code will need to code external or internal component. you will need to
refactor that code in specific method that you will overwrite for your
test.



class F {
       Object _f1(Object... args) {


               // used to be inline
               Object extInf = getExtInfo();
       }


       Object getExtInfo() {
        ....
       }
}

class FTest extends F {

       Object getExtInfo() {
        return new MockObject();
       }
}

If the object is to complex to Mock you can use the XStream to dump a
sample in uat environment.

5 - run the test !

6 - how to improve the solution

It is possible to create the list of test automatically by scanning the
samples directory e.g.. Making it easier to add sample and extend the
coverage of the test and behaviour.

This is an iterative process, it is not possible to make everything
testable in one go, take your time and surely step by step the quality
of your code will improve.

Dr. Horrible's Sing-Along Lyrics

2008-07-25T05:38:00.000-07:00

Here

The Java Spirit

2008-07-25T01:34:00.001-07:00

- Composition is better than inheritance
- Constructor are for initialization only
- destructor/finalizer are for debugging only
- Free resource explicitly
- Declare the interface not the type ie List myList; not ArrayList myList
- Optimize on metrics of a real system
- Use a proper development environment
- Don't believe the hype about closure, anonymous inner class cover 90% of the cases
- Properties are private or protected.
- Immutable is your friend.
- Don't pool object, we are in the 21st century
- Elegant code is code that is meaningful
- the most useful api are : Collections, common-lang, java.text

... To be continued

Too many open file descriptors, leak detection

2008-06-11T07:04:00.000-07:00

In Java it is good practice to manually free the resources such as Streams, Connection. This is the consequence of the non deterministic property of the Garbage Collector. You object that is not reference might wait minutes before being process - or might even never -, so you can rely and the destructor - the finalize method - to do that - the finalizer make the collection of the object really costly-.

So if you open a Stream you should ideally do that :


InputStream is = null
try {
  is = new FileInputStream(...);
  
  ...

} finally {
  if (is != null) {
    try { is.close(); ) catch(Exception e) { // IGNORE }
  }
}

That's a lot of boiler plate code, and it's probably a good idea to use a Template pattern for that - look af Spring JDBCTemplate for example -.

What happen if you don't do that? That depends on how many file you are gonna open. It's really likely that the object is gonna get GC and FileInputStream as a finalizer where it closed itself.

But if your application uses file intensively you might end up with "Too many open file descriptors" errors. If you're lucky you have few entry points where stream are open, so you can easily fix it. Otherwise you will need a way to take the part of the code that are erronous.

What I propose under is to use Javassist to instrument the FileInputStrean and FileOutputStream class.

First we need a class that we would inject into the FileXStream to capture the states and detect the leaks.


package io;

public class LeakDetector {
 
 private boolean closed;
 private Exception trace;
 
 
 public LeakDetector() {
  trace = new Exception();
 }
 
 public void close() {
  this.closed = true;
 }
 
 protected void finalize() {
  if (! closed) {
   printAlert();
  }
 }

 private void printAlert() {
  if (System.err != null) {
   System.err.println("LeakDetector detected a leak from the object open at the following trace :");
   trace.printStackTrace();
  }
 }
}

Then we need a class that instruments the FileXStreams


package io;

import java.io.IOException;
import java.util.concurrent.Callable;

import javassist.CannotCompileException;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtConstructor;
import javassist.CtField;
import javassist.CtMethod;
import javassist.NotFoundException;

public class Instrumenter implements Callable<Object> {

 private String[] clazzes;
 
 public Instrumenter(String[] args) {
  this.clazzes = args;
 }

 public static void main(String[] args) throws Exception {
  
  if (args == null || args.length == 0) {
   args = new String[] { "java.io.FileInputStream", 
     "java.io.FileOutputStream" };
  }
 
  new Instrumenter(args).call();
 }

 public Object call() throws NotFoundException, CannotCompileException, IOException {
  ClassPool pool = ClassPool.getDefault();
  for(String clazz : clazzes) {
   instrument(clazz, pool);
  }
  
  return null;
 }

 private void instrument(String clazz, ClassPool pool) throws NotFoundException, CannotCompileException, IOException {
  CtClass ctClass = pool.get(clazz);
  
  ctClass.addField(CtField.make("private io.LeakDetector __io_ld;", ctClass));
  
  for(CtConstructor cConst : ctClass.getConstructors()) {
   cConst.insertBefore("__io_ld = new io.LeakDetector();");
  }
  
  for(CtMethod cMethod : ctClass.getMethods()) {
   if (cMethod.getName().equals("close")) {
    cMethod.insertBefore("__io_ld.close();");
   } else if (cMethod.getName().equals("finalize")) {
    cMethod.insertBefore("__io_ld._finalize();");
   }
  }
  
  ctClass.writeFile("bin/java");
 }
}

This class write the new FileInputStream in the bin/java directory. We need do assemble the LeakDetector and the new FileInputStream, FileOutputStream under the same jar. all you need to next is to run your application with the "-Xbootclasspath/p:leakdetector.jar" on the command line.

Note: Applications that use this technique for the purpose of overriding a system class in rt.jar should not be deployed as doing so would contravene the Java 2 Runtime Environment binary code license.

If you run the following code, with the bootclasspath set :


import java.io.FileInputStream;
import java.io.FileOutputStream;


public class DodgyIOClass {

 public static void main(String[] args) throws Exception {
  
  
  new FileOutputStream("c:/test.txt");
  
  new FileInputStream("c:/test.txt");
  
  
  Thread.sleep(3000);
  System.gc();
  Thread.sleep(3000);
  
  
 }
}

It should prints :


LeakDetector detected a leak from the object open at the following trace :
java.lang.Exception
 at io.LeakDetector.<init>(LeakDetector.java:11)
 at java.io.FileOutputStream.<init>(FileOutputStream.java)
 at java.io.FileOutputStream.<init>(FileOutputStream.java:70)
 at DodgyIOClass.main(DodgyIOClass.java:10)
LeakDetector detected a leak from the object open at the following trace :
java.lang.Exception
 at io.LeakDetector.<init>(LeakDetector.java:11)
 at java.io.FileInputStream.<init>(FileInputStream.java)
 at java.io.FileInputStream.<init>(FileInputStream.java:66)
 at DodgyIOClass.main(DodgyIOClass.java:12)

JSR 203 NIO 2 in Java 7

2008-04-02T08:10:00.000-07:00

So it seems.

That would be probably one of the most awaited fix, a well design file system api.

no more


if (!file.delete())  {
 // delete did not work but I don't know why 
}

Java performance improvements touted

2008-03-28T09:22:00.000-07:00

Java is not the slowpoke of old days and performance now matches or exceeds applications developed in C, technologists stressed Wednesday during a presentation at TheServerSide Java Symposium in Las Vegas.

From : DZone

Nice to see that written again.

IE 6 display = 'none' not working

2008-03-14T03:01:00.000-07:00

see Here

Using


  element.style.display = ''

instead solve the problem

How to disable escaping in xslt

2008-03-14T02:48:00.001-07:00


 <xsl:text disable-output-escaping="yes"><!--</xsl:text>

NIO / IO debates

2008-03-05T08:53:00.000-08:00

More NIO / IO benchmark or evidence

Writing Java Multithreaded Servers - whats old is new (PDF Slides)

See Micro benchmark Channel vs InputStream / 2

String sorting tip

2008-01-24T01:25:00.001-08:00

Good tip here



 Collator esCollator =
   Collator.getInstance(new Locale("es"));
 Collections.sort(list, esCollator);

Image Resizer for widescreen picture frame and the main color problem

2008-01-05T04:13:00.000-08:00

-- UPDate
http://www.comesolvego.com/2008/04/29/resize-images-with-java-high-quality-and-working-solution/
--

For Xmas I receive a picture frame, unfortunately the image is expanded
to the size of the frame without respecting the ratio.

I try to find a batch solution to resize and fit the picture so that it
would not be deformed.

I did not find anything so I decided to try to write a small java
program to do it.
it would take the picture resize it so it fit in the frame and put it
center in a image the size of the frame.

It was relatively easy.


package org.srf;

import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.Rectangle;
import java.awt.RenderingHints;
import java.awt.image.BufferedImage;

public class FitImage {

 public BufferedImage  scaleImage(BufferedImage image, int width, int height) throws Exception {

  int originalHeight = image.getHeight();
  int originalWidth = image.getWidth();
  
  
  double scale = Math.min(
    height/ (double)originalHeight,
    originalWidth/ (double)originalWidth
  );
  
  int newWidth = (int)Math.round(scale * originalWidth);
  int newHeight = (int)Math.round(scale * originalHeight);
  
  
  // Draw the scaled image
  BufferedImage thumbImage = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB);
  
  Graphics2D graphics2D = thumbImage.createGraphics();
  graphics2D.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
    RenderingHints.VALUE_INTERPOLATION_BILINEAR);
  
  graphics2D.drawImage(image, 0, 0, newWidth, newHeight,
    null);

  return thumbImage;

 }
 
 public BufferedImage frameImage(BufferedImage image, int width, int height, Color backgroundColor) throws Exception {
  
  int paddingX = (width - image.getWidth()) / 2;
  int paddingY = (height - image.getHeight()) / 2;

  
  // Draw the scaled image
  BufferedImage thumbImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
  
  
  Graphics2D graphics2D = thumbImage.createGraphics();
  
  graphics2D.setColor(backgroundColor);
  
  graphics2D.fill(new Rectangle(0, 0, width, height));
  
  graphics2D.drawImage(image, paddingX, paddingY, null);

  return thumbImage;

 }
}

Then what color do I want to draw the border. The average color of the
picture? does not seems right if you have a picture with blue and green
you might end up with a yellow ...

So what about the dominant colour ? the main colour of the picture ?
That quite a fusy concept like that. The color are distributed in a 4
dimension space (red, green, blue, alpha) what would that mean to be the
dominant color? What would be an efficient algorythm to determine it.

I came up with something that give relatively good result also not sure
about the validity of it.

It iteratively cut the spaces in 2^4 boxes and keep the one where there
is most color.
It does the same thing on this box reduce it a specific number of time.
As the color are define by a number between 0-256, each iteration
dividing the solution possibilite by to, in 8 iteration I arrived to a
result. What does it mean not sure... but on test I did it seems to work
fine.


package org.srf;

import java.awt.Color;
import java.awt.image.BufferedImage;

public class ColorHelper {
 private boolean discardPixelOutOfTheBox = true;
 private boolean shiftRange = false;
 
 
 public void setDiscardPixelOutOfTheBox(boolean discardPixelOutOfTheBox) {
  this.discardPixelOutOfTheBox = discardPixelOutOfTheBox;
 }


 public void setShiftRange(boolean shiftRange) {
  this.shiftRange = shiftRange;
 }


 public Color getMainColor(BufferedImage image, int refineTo) {
  Color c = null;
  
  int[] rgbs = image.getRGB(0, 0, image.getWidth(), image.getHeight(), null, 0, image.getWidth());
  
  
  int[][] colorRanges = new int[][]{ {0, 0xFF}, {0, 0xFF},{0, 0xFF}, {0, 0xFF}};
  
  for(int refine = 0; refine < refineTo ; refine++) {
   int[] colorSpace = new int[2 * 2 * 2 * 2];
   for(int rgb : rgbs) {
    int gi = calculateSpacePoint(colorRanges, rgb);
    if (gi != -1 ) {
     colorSpace[gi] ++;
    }
   }
   
   int ci = calculateSpacePointWinner(colorSpace);
   
   shrinkSpace(colorRanges, ci);
   
  }
  
  int[] crgbs = new int[4]; 
  // take middle of colorRange
  for(int i = 0; i < colorRanges.length; i++) {
   int[] range = colorRanges[i];
   crgbs[i] =  (range[0] +  range[1]) /2  ;
  }
  
  c = new Color(crgbs[2],crgbs[1],crgbs[0],   crgbs[3]);
  
  return c;
 }


 private int calculateSpacePointWinner(int[] colorSpace) {
  int ci = 0;
  int ciValue = 0;
  for(int i = 0; i < colorSpace.length; i++) {
   if (colorSpace[i] > ciValue) {
    ci = i;
    ciValue = colorSpace[i];
   }
  }
  return ci;
 }


 private int calculateSpacePoint(int[][] colorRanges, int rgb) {
  int gi = 0;
  for(int i = 0; gi != - 1 && i < 4; i++) {
   int v  = (rgb >> (i * 8)) & 0xFF;
   
   if (!discardPixelOutOfTheBox 
     || (v >= colorRanges[i][0] && v <= colorRanges[i][1])) {
    int rangeMiddle = (colorRanges[i][0] + colorRanges[i][1]) / 2;
    
    int vi = v < rangeMiddle ? 0:1;
    
    gi |=  vi << i;
   } else {
    gi = -1;
   }
   
  }
  return gi;
 }


 private void shrinkSpace(int[][] colorRanges, int ci) {
  // re-adjust colorRange
  for(int i = 0; i < colorRanges.length; i++) {
   int[] range = colorRanges[i];
   int cri = (ci >> i ) & 0x1;
   
   shrinkDimension(range, cri);
  }
 }


 private void shrinkDimension(int[] range, int cri) {
  int rangeShift = shiftRange  ? (range[1] - range[0]) /4 : 0;
  int middleRange = (range[0] +  range[1]) /2;
  if (cri == 0) {
   range[1] =  middleRange + rangeShift ;
  } else {
   range[0] = middleRange - rangeShift;
  }
 }

}

XML/Castor notes 1

2007-12-11T01:38:00.000-08:00

Marshal to a document object



 Document serializeObject(Object o) throws Exception {

  DocumentBuilder documentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
  
  Document document = documentBuilder.newDocument();
  
  Marshaller marshaller = new Marshaller(d);
  
  synchronized (mapping) {
   marshaller.setMapping(mapping);
  }
  
  marshaller.marshal(o);
  
  return document;
 }

Write the Document to an xml file



 void write(Document doc, File f) throws Exception {
  Transformer trans = TransformerFactory.newInstance().newTransformer();
  
  Source source = new DOMSource(doc);
  Result result = new StreamResult(new FileOutputStream(f));
  
  trans.transform(source, result);
  
 }

Acegi 2

2007-12-10T04:34:00.001-08:00

does not kill fairy by murder anymore.

Map & Reduce explained for dummies

2007-12-10T03:00:00.000-08:00

Can Your Programming Language Do This?

Also I'm quite sure the cost of doing it in java is overrated - what about anonymous class, used in the common collection for example - ... This is a nice piece of explanation.

Interruptible Call

2007-12-06T07:34:00.001-08:00

In the not so new concurrent api, a call - Executor.call(new Call()) can be cancel by the Future object return.

Future.cancel(true) is gonna interrupt the thread. But does that mean that your code gonna stop automatically if it already started? Only if it's on a blocking call, which it's quite unlikely. So that your code is interruptible you need to test it the thread has been interrupted.


   while(test) {
      if (Thread.interrupted()) {
         throw new InterruptedException();
      }
   }

This way your cancel is more reactive.

Micro benchmark Channel vs InputStream / 2

2007-12-04T07:40:00.000-08:00

If you follow actually the samples like Grep. Channel are slower than the BufferedReader

On a directory with 16 files, with a total of 42 MB :
nio direct=Time Taken: 3703ms,
nio=Time Taken: 3781ms,
io=Time Taken: 3063ms

So ?


 public class NIOReader implements Reader {

  private ByteBuffer byteBuffer;
  
  public NIOReader(int bufferSize, boolean direct) {
   if (direct) {
    this.byteBuffer = ByteBuffer.allocateDirect(bufferSize);
     
   } else {
    this.byteBuffer = ByteBuffer.allocate(bufferSize);
   }
   
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   FileChannel channel = null;
   
   try {
    fis = new FileInputStream(f);
    
    channel = fis.getChannel();
    
    MappedByteBuffer buffer = channel.map(MapMode.READ_ONLY, 0, channel.size());
    
    CharBuffer charBuffer = Charset.defaultCharset().decode(buffer);

    return charBuffer.toString();
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
  }

 }

Micro benchmark Channel vs InputStream

2007-12-04T02:27:00.000-08:00

I wrote a little non representative test to check is it was worth rewriting the IO, in an application that write and read a lot of data from the file system, using the not so new NIO api.

On a directory with containing 8 305 files all around 1 to 4 KB the result are around :
nio direct=Time Taken: 5610ms
nio=Time Taken: 5766ms
io=Time Taken: 5578ms

On a directory with 16 files, with a total of 42 MB :
nio direct=Time Taken: 875ms
nio=Time Taken: 906ms
io=Time Taken: 3079ms

So for big files it's definitly worth it, but not for small file.

It's tested under windows XP, would the result be the same under a Unix OS? Don't know ...



import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.HashMap;
import java.util.Map;

import junit.framework.TestCase;

import org.apache.log4j.Logger;

public class NIOTest  extends TestCase {
 
 private static Logger  logger= Logger.getLogger(NIOTest.class);
 public class NIOReader implements Reader {

  private ByteBuffer byteBuffer;
  
  public NIOReader(int bufferSize, boolean direct) {
   if (direct) {
    this.byteBuffer = ByteBuffer.allocateDirect(bufferSize);
     
   } else {
    this.byteBuffer = ByteBuffer.allocate(bufferSize);
   }
   
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   FileChannel channel = null;
   StringBuilder sb = new StringBuilder((int)f.length());
   
   try {
    fis = new FileInputStream(f);
    
    channel = fis.getChannel();
    
    
    while(channel.read(byteBuffer) != -1) {
     byteBuffer.flip();
     sb.append(byteBuffer.toString());
     byteBuffer.clear();
    }
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
   return sb.toString();
  }

 }
 public class IOReader implements Reader {

  private byte[] buffer;

  public IOReader(int bufferSize) {
   this.buffer = new byte[bufferSize];
  }

  public String readFile(File f) throws IOException {
   FileInputStream fis = null;
   
   StringBuilder sb = new StringBuilder((int)f.length());
   
   try {
    fis = new FileInputStream(f);
    
    int c = -1;
    
    while( (c = fis.read(buffer)) != -1) {
     sb.append(new String(buffer, 0 , c));
    }
    
   } finally {
    if (fis != null)
     fis.close();
   }
   
   
   
   
   return sb.toString();
  }

 }

 File dir = new File ("C:\\my directory with files");
 
 int size = 3;
 int buffer = 4096;
 
 public void testIO() throws IOException {
  for (int i = 0 ; i < size; i++) {
   Map stopWatches = new HashMap();
   
   logger.info("test io");
   stopWatches.put("io", testLoop(size, new IOReader(buffer), dir));
   logger.info("test nio");
   stopWatches.put("nio", testLoop(size, new NIOReader(buffer, false), dir));
   logger.info("test nio direct");
   stopWatches.put("nio direct", testLoop(size, new NIOReader(buffer, true), dir));
   
   
   logger.info(stopWatches.toString());
  }
 }
 
 public StopWatch testLoop(int size, Reader reader, File dir) throws IOException {
  StopWatch stopWatch = new StopWatch();
  
  stopWatch.start();
  
  for(int i = 0; i < size; i ++) {
   test(reader, dir);
  }
  
  stopWatch.stop();
  
  logger.info("end loop " +stopWatch);
  return stopWatch;
  
 }
 
 public void test(Reader r, File dir) throws IOException {
  for(File f: dir.listFiles()) {
   if (f.isFile()) {
    String str = r.readFile(f);
    //logger.debug("read "+ str.length()+" in "+f);
   }
  }
 }
 
 interface Reader {
  String readFile(File f) throws IOException;
 }

}

7.4 Preventing Castor from checking for a default constructor (since 0.9.5)

2007-11-29T09:39:00.001-08:00

Sometimes it's useful to prevent Castor from checking for a default constructor, such as when trying to write a mapping for an interface or type-safe enum. You can use the "undocumented" verify-constructable="false" attribute on the element to prevent Castor from looking for the default constructor.

been looking for this one ...
See

Interesting post ab out Database Partitioning using Spring

2007-11-28T01:36:00.000-08:00

Horizontal Database Partitioning with Spring and Hibernate

Castor elusive StackOverflowError

2007-11-01T06:10:00.000-07:00

On one of my project, I'm using Casor - 1.1.2.1 - to serialize an object tree to xml. At the beginning of the week I had a weird StackOverflowError when writing a big set of objects. I did not pay much attention do it, was not a priority.

Today I had the same error, on a very reasonably size object tree. So I try to find where it was comming from - a cycling mapping maybe ... -. I putted my eclipse in debug mode, and add a breakpoint on the error. And then no more SOE ... but a error when setting the mapping in the Marshaller... weird ...

What can it be? that looks like a multi thread issue... But the Marshaler are both distinct, the only common thing is the Mapping. Could it be ?
So I replaced


 Marshaller marshaller = new Marshaller(serializer
   .asDocumentHandler());
 if (mapping != null) {
     marshaller.setMapping(mapping);
 }
 marshaller.marshal(o);

with


 Marshaller marshaller = new Marshaller(serializer
   .asDocumentHandler());
 if (mapping != null) {
  synchronized (mapping) {
   marshaller.setMapping(mapping);
  }
 }
 marshaller.marshal(o);

And it solve my problem.

Conclusion
Castor Mapping object is not Thread Safe...

How weird....

Comparing double with relative precision

2007-09-27T02:11:00.001-07:00


  private static final int DOUBLE_PRECISION = Math.getExponent(0.00000000000001);

 
  public static boolean equalsDouble(double d1, double d2) {
    double diff = Math.abs(d1 - d2);

    if (diff == 0.0) {
      return true;
    }
  
    int minExp = Math.min(Math.getExponent(d1), Math.getExponent(d2));
  
    int diffExp = Math.getExponent(diff);
  
    int relExp = diffExp - minExp;
  
    return  relExp <= DOUBLE_PRECISION;
 }

The test :

 
  public void testDoubleEquals() {
    assertTrue( Utils.equalsDouble(0.0, 0.0));
    assertTrue( Utils.equalsDouble(123.56, 123.56));
    assertTrue( Utils.equalsDouble(15698965.23659898, 15698965.23659898));

    assertTrue( Utils.equalsDouble(1.000000000000001, 1.000000000000000));
    assertTrue( Utils.equalsDouble(1000000000000001d, 1000000000000000d));
    assertTrue( Utils.equalsDouble(0.000000000000001000000000000001d, 0.000000000000001d));
  
    assertFalse( Utils.equalsDouble(1.00000000000011, 1.00000000000001));
    assertFalse( Utils.equalsDouble(100000000000011d, 100000000000001d));
    assertFalse( Utils.equalsDouble(0.00000000000000100000000000011d, 0.00000000000000100000000000001d));

    assertFalse( Utils.equalsDouble(15d, 1d));
    assertFalse( Utils.equalsDouble(15d, 15.6d));

 }

Cursors random walks

2007-09-20T02:24:00.000-07:00

I was working on an existing database with a simple table like this


CREATE TABLE T (
 id int,
 name varcher(255),
 job_id int
)

there is about 400 rows in this table, and the DB design is quite poor.

I decided to refactor a procedure that was going through the table T using a counter and assuming that it would match the id. It was working fine but it's kind of hard to imagine a weirder way of parcouring a table.

It did look like this


  select @insertRowCount = count(*) from T
    
  while @counter < @insertRowCount
  begin
   select @name = name from T where id = @counter
   
   ...
   
   set @counter = @counter+1
  end

The most obvious modification was using a cursor instead of this counter thing.


  declare Tcursor cursor for 
   select  name , id
   from T

  open jobInfo

  fetch jobInfo into  @name, @id

  while(@@sqlstatus=0)
  begin

   fetch jobInfo into  @name, @id
  end
  
  close jobInfo

Looks better, doesn't it?

except that the cursor did not return all the row ... Kind of weird.

After a bit of googling I find my answer, it need a unique index, which you've got normaly with a primary key...


  create unique index PK_T on T(id)

PS: I do know such database is badly design...

Micro benchmark - Collection sorting

2007-08-24T05:17:00.000-07:00

What is the most efficient



    List list = new ArrayList();
    
    for(Long l : array) {
     list.add(l);
    }
    
    Collections.sort(list);



    TreeSet list = new TreeSet();

    for(Long l : array) {
     list.add(l);
    }

After a micro benchmark, on 1000 random arrays of 100 000 element

	Total Time (ms)	Average Time (ms)	Min Time (ms)	Max Time (ms)
Collections.sort	81 893	81	62	125
TreeSet	187 578	187	140	281

In the Benchmark the Collections.sort takes 43 % of the time that it takes using the Tree.

Normalization under attack

2007-08-16T01:33:00.001-07:00

Data normalization, is it really that good?

Xalan-specific output properties nightmare

2007-07-02T06:48:00.000-07:00

At work we are using Xalan 2.7.0, it works fine except that for some reason we would like to modify a bit the html output method. This method strip the attribute value for readonly and disabled attribute. E.g :

is output as

that's fair enough but not the behaviour we need. So I start looking for a way to change that. I quicly found the way the method influence the Serializer. If you like in the output_html.properties you'll find

# Xalan-specific output properties. These can be overridden in the stylesheet
# assigning a xalan namespace. For example:
#
# xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
# xmlns:xalan="http://xml.apache.org/xalan">
#
# xalan:content-handler="MyContentHandler"/>;
# ...
# Note that the colon after the protocol needs to be escaped.

That was a great start. So how does the html content handler work ? Would it be possible to easily change its behaviour ? At first I thought so. The code interesting here is in the method protected void processAttribute(java.io.Writer writer, String name, String value, ElemDesc elemDesc) throws IOException

/**
   * Process an attribute.
   * @param   writer The writer to write the processed output to.
   * @param   name   The name of the attribute.
   * @param   value   The value of the attribute.
   * @param   elemDesc The description of the HTML element
   *           that has this attribute.
   *
   * @throws org.xml.sax.SAXException
   */
  protected void processAttribute(
      java.io.Writer writer,
      String name,
      String value,
      ElemDesc elemDesc)
      throws IOException
  {
      writer.write(' ');

      if (   ((value.length() == 0) || value.equalsIgnoreCase(name))
          && elemDesc != null
          && elemDesc.isAttrFlagSet(name, ElemDesc.ATTREMPTY))
      {
          writer.write(name);
      }
      else
      {
          // %REVIEW% %OPT%
          // Two calls to single-char write may NOT
          // be more efficient than one to string-write...
          writer.write(name);
          writer.write("=\"");
          if (   elemDesc != null
              && elemDesc.isAttrFlagSet(name, ElemDesc.ATTRURL))
              writeAttrURI(writer, value, m_specialEscapeURLs);
          else
              writeAttrString(writer, value, this.getEncoding());
          writer.write('"');

      }
  }

So when the ElemDesc which is defined in the ToHtmlStream got the attribute with the fllag ATTREMPTY it does that. Easy ! so we probably can extend this class and change the behaviour of processAttribute? No! Is there an easy way to change the ElemDesc? No! Maybe we can just copy and past the class? Not really without putting it in the jar, solution a bit to risky because the Xalan Jar is probably shared.

So the solution? using reflexion to do a deep copy of the specific not very friendly Trie object - we want that only on the specified page -, and create an Handler that gonna delegate to the classic ToHtmlStream to which we are gonna change the root of the Trie object to point to the copy with the modified ElemDesc ... It does work but it's a nightmare of 200 lines of reflexion that used the name of private field of private inner class ...


public class NewToHTMLStream implements SerializationHandler {

    ToHTMLStream stream = null;

    Object trieCopy = null;

    static final Set ATTRIBUTES = new HashSet(Arrays.asList(new String[] { "READONLY", "DISABLED" }));

    static final Object lock = new Object();

    private static Field fieldm_htmlInfo;

    private static Field field_trie_root;

    private static Field field_trie_node_nodes;

    private static Field field_trie_node_value;

    private static Field field_m_elementFlags;

    private static Field field_ElemDesc_m_attrs;

    private static Field field_ElemDesc_m_Flags;

    private static Method elementDesc_setAttr;

    private static Constructor eltDescConstructor;

    private static Constructor nodeConstructor;

    private static Constructor constructorTrie;

    private static Field fieldMMap;

    private static Field fieldMValues;

    private static Field fieldMFirstFree;

    public NewToHTMLStream() throws IllegalArgumentException, NoSuchFieldException, IllegalAccessException, SecurityException, NoSuchMethodException, InvocationTargetException, InstantiationException {

        stream = new ToHTMLStream();

        synchronized (lock) {
            if (trieCopy == null) {
                trieCopy = copyTrie();
            }
        }

        
        Object htmlInfo = fieldm_htmlInfo.get(stream);
        
        field_trie_root.set(htmlInfo, field_trie_root.get(trieCopy));

    }

    public static Object copyTrie() throws NoSuchFieldException, IllegalArgumentException, IllegalAccessException, SecurityException, NoSuchMethodException, InvocationTargetException, InstantiationException {
        initReflexion();

        Object m_elementFlags = field_m_elementFlags.get(ToHTMLStream.class);
        Object newTrie = constructorTrie.newInstance(new Object[] { m_elementFlags });

        Object root = (Object) field_trie_root.get(m_elementFlags);

        Object newRoot = copyNode(root, newTrie);

        field_trie_root.set(newTrie, newRoot);

        return newTrie;

    }

    private static Object copyNode(Object root, Object trie) throws IllegalArgumentException, InstantiationException, IllegalAccessException, InvocationTargetException {
        Object copy = nodeConstructor.newInstance(new Object[] { trie });

        field_trie_node_value.set(copy, copyElementDescription(field_trie_node_value.get(root)));

        Object[] originalNodes = (Object[]) field_trie_node_nodes.get(root);
        Object[] newNodes = (Object[]) field_trie_node_nodes.get(copy);

        for (int i = 0; i < originalNodes.length; i++) {
            Object n = originalNodes[i];
            if (n != null) {
                newNodes[i] = copyNode(n, trie);
            }
        }

        return copy;
    }

    private static ElemDesc copyElementDescription(Object o) throws IllegalArgumentException, InstantiationException, IllegalAccessException, InvocationTargetException {
        if (o == null)
            return null;

        ElemDesc desc = (ElemDesc) o;

        ElemDesc desc2 = eltDescConstructor.newInstance(new Object[] { field_ElemDesc_m_Flags.get(desc) });

        /*
         * m_attrs copy except ATTRIBUTES
         */

        StringToIntTable m_attrs = (StringToIntTable) field_ElemDesc_m_attrs.get(desc);

        if (m_attrs != null) {
            String[] m_map = (String[]) fieldMMap.get(m_attrs);
            int[] m_values = (int[]) fieldMValues.get(m_attrs);
            int m_firstFree = (Integer) fieldMFirstFree.get(m_attrs);

            for (int j = 0; j < m_firstFree; j++) {
                int flag = 0;
                String attributeName = m_map[j];
                if (!ATTRIBUTES.contains(attributeName)) {
                    flag = m_values[j];
                }

                elementDesc_setAttr.invoke(desc2, new Object[] { attributeName, flag });
            }
        }

        return desc2;
    }

    private static void initReflexion() throws NoSuchFieldException, NoSuchMethodException {
        fieldm_htmlInfo = setAccessible("m_htmlInfo", ToHTMLStream.class);

        field_m_elementFlags = setAccessible("m_elementFlags", ToHTMLStream.class);

        constructorTrie = field_m_elementFlags.getType().getDeclaredConstructor(new Class[] { field_m_elementFlags.getType() });
        constructorTrie.setAccessible(true);

        field_trie_root = setAccessible("m_Root", field_m_elementFlags.getType());
        field_trie_node_nodes = setAccessible("m_nextChar", field_trie_root.getType());
        field_trie_node_value = setAccessible("m_Value", field_trie_root.getType());

        nodeConstructor = field_trie_root.getType().getDeclaredConstructor(new Class[] { field_m_elementFlags.getType() });
        nodeConstructor.setAccessible(true);

        field_ElemDesc_m_attrs = setAccessible("m_attrs", ElemDesc.class);
        field_ElemDesc_m_Flags = setAccessible("m_flags", ElemDesc.class);
        elementDesc_setAttr = setAccessible("setAttr", new Class[] { String.class, Integer.TYPE }, ElemDesc.class);

        eltDescConstructor = ElemDesc.class.getDeclaredConstructor(new Class[] { Integer.TYPE });
        eltDescConstructor.setAccessible(true);

        fieldMMap = setAccessible("m_map", StringToIntTable.class);
        fieldMValues = setAccessible("m_values", StringToIntTable.class);
        fieldMFirstFree = setAccessible("m_firstFree", StringToIntTable.class);
    }

    private static Method setAccessible(String methodName, Class[] paramTypes, Class clazz) throws SecurityException, NoSuchMethodException {
        Method method = clazz.getDeclaredMethod(methodName, paramTypes);

        method.setAccessible(true);
        return method;
    }

    private static Field setAccessible(String fieldName, Class clazz) throws NoSuchFieldException {
        Field fieldMAttrs = clazz.getDeclaredField(fieldName);

        fieldMAttrs.setAccessible(true);
        return fieldMAttrs;
    }

    /**
     * DELEGATE to stream ...
     */

Wait for me !

2007-03-12T09:00:00.000-07:00

In the J2EE world you're not suppose to play with Thread. Some people are even afraid just by the name of it. It's easy to understand why, there are a lot of issues to be aware of if you're using them, security, synchronization, pooling, class loading. Then it's easy to discard threading subjects as irrelevant to web application.

There is nothing more wrong in my opinion. Because in a J2EE application thread are everywhere, a HTTP Request is served in a Thread, a MessageListener run in a Thread. The spring MessageContainers run n Thread. Etc ... You can't avoid thinking about that. A classic example is in the Old Struts ...

But it also be really useful to do some integration testing. If you are lucky to use the new Spring 2 message container bean, that enable you to have a MessageDrivenBean without an EJB container. All you have to do is implementing the MessageListener interface and a bit of configuration.

This Listener gonna then be called on the container when a message arrive.

Now imagine that for integration test purpose you want to know when then listener has been executed.

Here you have two strategies, The I come every n seconds looking if you are finished and The I'm gonna have a nap, just wake me up when you're here.

Let's write a method waitUntilIAmHere on the listener, that the test will call

listener.waitUntilIAmHere();

The I come every n seconds looking if you are finished

or the stalker one. A loop check every so and then for the state of the object until it changes.

    boolean weAreThere;

   public void waitUntilYouAreHere() throws InteruptedException {
      while(listener.areWeThereYet()) {
          Thread.sleep(sleepTime);
        }
   }

   public boolean areWeThereYet() throws InteruptedException {
       synchronized(this) {
           return weAreThere;
       }
   }

   public void onMessage(Message m) {
       doMyThing();

       iAmHere();
   }

   void iAmHere() {
       synchronized(this) {
           weAreThere = true;
       }
   }

That might be OK for a test but it's far from being elegant, and probably far from being efficient.

The I'm gonna have a nap, just wake me up when you're here

uses some rendez-vous api, wait and notify. Wait sleep the Thread until the object on which it's waiting sends a notification. Notify sends the notification. In this case we are using notifyAll in case two threads are waiting. One thing to be aware with notify is as stated in the documentation : The choice [of which thread is awakened] is arbitrary and occurs at the discretion of the implementation.

    boolean weAreThere;

   public void waitUntilYouAreHere() throws InteruptedException {
       synchronized(this) {
           if (!weAreThere) {
               this.wait(timeout);
           }
       }
   }

   public void onMessage(Message m) {
       doMyThing();

       iAmHere();
   }

   void iAmHere() {
       synchronized(this) {
           weAreThere = true;
           this.notifyAll();
       }
   }

This one is in number of effective line as complex as the precedent, except that it's much cleaner. The interruption is easily deal with, so is the timeout.

wait and notify are key methods to work with Thread, if you look in any Thread Pool implementation they more than probably deal with the thread waiting for a new task to execute.

Some Resources :

Java theory and practice: Dealing with InterruptedException

Object.wait()

Object.notifyAll()

New Google Mail feature : the mail fetcher

2007-03-02T08:51:00.000-08:00

If you're like me and got more than one email account, you're gonna appreciate this new functionality ! no need to check all the different web mail, google now offer the possibility to fetch your email for you.

Autowire is not your friend

2007-02-22T01:17:00.000-08:00

Imagine you have an application that is working fine. It uses spring as a DI container and for the AOP nothing extraordinary except that for some reason the default-autowire is set to "autodetect". Part of the config is the following, a MBean exporter that take a map as a property - JMX is really nice integrated in spring, doc here - .

Weeks later somebody makes some modification, and everything breaks. He just had a Properties in the config :

And now MBeanExporter complain that the map does not contains the proper kind of object.

So what happened here ?

The response is simple, the autowiring overwrite - I yes i think it's a strange behaviour - the map of the exporter with the environmentProperties - Properties extends Hashtable implements Map -. Fortunately you can had an attritbute to the bean autowire="no" to save the day, but my feeling is that you should not activate auto wiring by default as recommanded I think but documentation ?

Cruise control label incrementor

2007-02-20T04:22:00.000-08:00

I wanted to tag cvs after each succesful build in cruise. All you need to do is add a task to add a task a the end of the build script :


 <target name="cvstag" if="label">
  <cvs command="tag ${label}" quiet="false" reallyquiet="false" />
 </target>

label is set by cruisecontrol. You need to add the labelincrementor plugin. You can miss this point as it is the default one though.



        <plugin name="labelincrementer"
         classname="net.sourceforge.cruisecontrol.labelincrementers.DefaultLabelIncrementer"/>

...

and in your project configure the defaultLabel and the separator - must be change to '-' -


...
    <project name="myproject">
        <labelincrementer defaultLabel="cruise_${project.name}-1" separator="-"/>

...

If the tag already exists it's not gonna move it.

Last point to reset the label in your project, the only way I found was to delete the {project.name}.ser and it's gonna restart at the defaultLabel.

The PermGen daemon.

2007-02-20T00:21:00.000-08:00

The life of Java Web application developers is most of the time a quiet and smooth road, on which minor hole makes the trip exiting. Until you end up facing some big ugly monster, the OutOfMemory ones are part of the very nasty. There is 3 types of OutOfMemory monster:
- the stack OutOfMemory, also known as the classic. It can be due to a memory leak or to the dwarfish heap, which I've rarely seen this days. Check your code and get a normal heap.
- the native stack OutOfMemory, this one is nastier than the precedent, also far less common. It happens when some native code failed to release memory. Oracle App Server used to have one when you had link to some nonexistent file - nice fail safe ... -.
- and the 3rd one and the one of interest here, the PermGen daemon. It introduces it self with a loud and devastating "OutOfMemoryError: PermGen space". This one is probably due to a nasty daemon that keep the ClassLoader in its grip.

First what is this PermGen ? The PermGen is a specific place of the java memory used to store "permanent object", like class definition. There is more info in what is it specifically elsewhere this is an enough approximation for the story that follow.

So when you deploy your web app into your web container - Tomcat for example - this webapp is load in memory, the classes definition are loaded too. The classes definition can be Garbage Collected and they will when their ClassLoader gonna be collected. That happen normally when there is no more reference to the class loader, which is when there is no more class instantiated by it. That's what normally happen when you undeploy or stop your application, freeing all the memory used by your impressive work of art.

But sometimes for a reason known only by yourself - asynchronous processing, report, show off, ... - you use a Thread in your application to do some stuff. Check that the database is up, or count the time, or whatever it is it's running. And the thread it does not really know about the web app and when the web app stop are undeploy. So To avoid any problem the shutdown of your application server - killing it is harder that just a quick Ctrl-C - you set the thread as daemon. Meaning that when the app server stop the thread gonna stop what it was doing by whatever magic it is - so stop is deprecated but setDeamon which in fine gonna do something equivalent to a stop is not ... -.

When you're developing you don't really won't to stop and start your tomcat each time you deploy, you probably got a neaty script that undeploy and deploy the application for you ! Or you one of this heretics who believe in peace not war and use a exploded web app. Every time your touching your web.xml, your web-app restart magically.

This work fine, except if you introduce the daemon in your webapp. After 2, 3 maybe 4 restart your web server shout in despair "OutOfMemoryError: PermGen space" !

What happened ? It works fine for days, it's only when I redeploy ! That's the work of the PermGen Daemon. And here is what happens :
- My web app v1 starts, my Thread named daemon1 - it's quite comfy to name your thread - are her dong there work. The classes definition, the class loader, and deamon1 all happy in the play ground.
- Developer after improving dramatically the design of his application wants to update the version in his Tomcat. Touch down, undeploy, deploy here it goes.
web app v1 stops, daemon1 does not give a shit, class loader still reference by daemon 1. web app v2 starts. with daemon 2 and a new class loader

So now you've got two daemon, 2 classloader that contains almost the same class definitions. you do that another couple of time and here is you permgen error.

So please, please when the web app stop make you thread stop too - that also means the Executor in java 5 -. This problem is quite common, the Active MQ connection factory has that in the 4.0.2 and probably 4.1.0 ... It's easy to ignore this error, and blame web container bug, but it's not. I'd guess the main problem is this notion of daemon Thread ...

London Java Meetup - 27 Feb 2007

2007-02-19T09:17:00.000-08:00

Never been there yet, but gonna have a couple of guinness with London Geeks.

some geeks, some beer, some rain...

Hibernate 3.2 broke the custom types

2007-02-15T06:10:00.000-08:00

A repost from my old blog :

This change on hibernate 3.2 can cause hibernate to not detect change in custom type on a collection property. e.g. if you want to map a comma separated list of string to a Set :

"String1, String2, String3" <=> { "String1", "String2", "String3" }.

After an update the loadedState gonna reference the set in the entity. So a change in this Set by the application gonna change also the Set in the loaded state as they are the same. So hibernate is not gonna be able to check if the Entity is dirty properly.

Solution stay in 3.1 or mark your object as hasUpdateGeneratedProperties, which might not be good for performances ...

I reported the Issue posted, still no change...

Welcome

2007-02-14T01:57:00.000-08:00

At that end of the cyber space, here come a new node. It's gonna be mainly about Java and other IT stuff. What I need to write about :
- how to deal with Thread in a web app, and what happens when it's not done properly.
- some stuff about thread management and how it can be useful in your test
- JMX and lo4j
- the beauty of XA Transaction
- JMS is my friend, but redilivery policy does not seems to work properly in active mq
- how struts 2 is wonderful and JSF sucks :)
- should I reconcile myself with maven
- cruise control easy

that's some ideas ...