Ronald Servant

Finalizers are still bad, don't use them

2011-07-13T08:44:00.001-07:00

There have been a couple of recent posts to remind us that finalizers in Java are bad. Unfortunately, sometimes they are required. Usually because you are using a framework that uses them.

One from Peter Burka on a reachability issue in Java that also touches on finalizers. This post has good discussion going on in the comments.

The second is a developerWorks article on a finalizer vulnerability. It describes how a finalizer attack works and how to write your code to secure yourself against it. Well worth a read.

Anti-Pattern: Java getter

2011-05-18T12:29:00.000-07:00

I recently fixed a 'bug' in the IBM JDK where running the following Groovy code would yield an unexpected result.

Consider the following Groovy snippet:

println AClass.aVariableName

Now consider this Java code:

public class AClass {
static int aVariableName = 1;
public int getAVariableName() {
return aVariableName;
}
}

The result of running the Groovy code will be to return 1.

What if we changed the Java code to:

public class AClass {
static int aVariableName = 1;
public int getAVariableName() {
if (aVariableName < 1) return aVariableName;
return 10;
}
}

What would the result of running the Groovy code be? What would you want it to be?

The result in this case would be 1. The desired result is likely 10 (or at least that was the desired result in the case of the 'bug' I fixed).

How does this happen? I am no Groovy expert, but if I were to guess, it would appear that Groovy first tries to match aVariableName to a variable on AClass. In this case it finds one and therefore returns the value of that variable. This is most likely accomplished using reflection, so using a security manager could make this illegal.

Now let's consider this final Java code snippet:

public class AClass {
private static int aPrivateVariableName = 1;
public int getAVariableName() {
if (aPrivateVariableName < 1) return aPrivateVariableName ;
return 10;
}
}

What would the result of running the Groovy code be?

In this case it returns 10, as I hoped would be from the earlier version of the Java code.

Again not a Groovy expert, but it appears that when Groovy doesn't find a variable of the appropriate name on the class it will then look for a getter. In this case it finds getAVariableName() and will invoke it.

So what is the anti-pattern? Don't create what looks like a getter method for a variable if it is in fact NOT a simple getter. The code in this case does some processing and may simply return the variable or perhaps something else entirely.

Is this really an anti-pattern for Java programming though?

It could be argued that because the variable is private we should be able to name it whatever we want. The ambiguity comes in when new languages, like Groovy, start being implemented in Java. Groovy has, rightly or wrongly, made it exceedingly easy to reach for that private variable.

Should Groovy be changed to look for the 'getter' method first? Maybe. But that doesn't change the fact that something that looks like a getter isn't one.

We have all been taught, I hope, to be careful when naming our variables and methods. This is just a new challenge.

Using Eclipse MAT to Analyze a Large Heap Dump

2011-03-10T12:33:00.001-08:00

I recently had a large heap dump that I needed to analyze for potential leaks or other patterns that could lead to poor performance. I wanted to use Eclipse Memory Analyzer (MAT), which you can get bundled in IBM Support Assistant (ISA) or as a standalone application.

The size of the heap dump I needed to analyze was 12GB and it was a challenge to get processed. My windows desktop with 8GB of RAM had a hard time (it locked up over the weekend trying). I had a bit better luck with my iMac, also with 8GB of RAM, but the machine was unusable the entire time the heap dump was being parsed.

The problem was that I was shooting my self in the foot on those platforms. I was setting the -Xmx of MAT to between 10GB and 20GB. If you actually use that much memory you will notice that a machine with only 8GB of RAM will start swapping itself to death. The iMac did in fact eventually finish doing the initial parse of the file (after about 15 or 16 hours). I did this over the weekend at home.

When I got back into the office on Monday and noticed that my desktop had locked up, I decided to hand the dump off to a colleague at work who has access to a set of dedicated servers for this purpose. The problem there is that the default heap sized used for MAT was ~5GB, which is too small when trying to analyze a 12GB heap dump. We wasted a day and couple of machines trying to parse the dump.

So now I was at 4 wasted days trying to analyze this rather large dump. We tried one more time with the 'dedicated' servers and got a parsed dump about 18 hours later, with I think an 8GB Xmx for MAT. Unfortunately, after downloading all the parsed data I realized that none of the standard reports had been run. So I still needed a machine I could load the parsed heap dump into and run the reports I was looking for and perhaps use some of the ad hock investigation tools MAT provides.

Now I did what I should have done the second I originally received this heap dump. I used the large linux box we had in the lab. It is a 64 core 64bit system with 64GB of RAM. I set the Xmx to 30GB for MAT and launched the application. It soaked up the parsed heap dump and ran the reports I wanted in minutes not hours. It was a fantastic experience and it allowed me to see how fantastic a tool MAT really is. MAT appears to be generally single threaded in its implementation, so lots CPU cores won't really help make it run faster. It is, however, memory hungry, so lots of RAM is a big bonus.

So here is my, I now feel obvious, recommendation: when running MAT on a heap dump you will want to set the Xmx for MAT to at least the size of the heap dump you are analyzing and, most importantly, run it on a machine that has at least as much RAM as the Xmx you specified.

For example, if the heap dump represents a Java heap of 20GB*** you will want to run MAT with an Xmx of 20GB or larger and run it on a machine that has 20GB of RAM or more.

*** The heap dump file is likely significantly smaller than the Java heap it represents. My 12GB heap dump was represented in a file that was 3.5GB in size.

UPDATE: There is a fantastic new article on developerWorks about MAT and it's various features. Well worth a read through.

UPDATE 2011/07/13: Another article on how to build yourself a 64bit standalone MAT with IBM extensions.

IBM is not abandoning Harmony

2010-10-13T13:32:00.000-07:00

With the recent announcement that IBM is joining the OpenJDK community it is now a widely held belief that IBM is abandoning the Apache Harmony project.

This is not the case.

IBM has shipped product that contains Harmony code (e.g., IBM SDK 6.0). IBM will support it's customers, that includes ensuring that the Harmony code we shipped is supported.

Will there be the same level of investment in Harmony by IBM as there was in the past? No, not likely, as IBM will shift the bulk of its resources in this area to OpenJDK. Harmony, however, will not be abandoned.

DISCLAIMER: These are my opinions only and not those of IBM.

IBM - Oracle working together on OpenJDK

2010-10-12T14:35:00.000-07:00

A quick round up of blog postings about the IBM - Oracle announcement

IBM joins the OpenJDK community, will help unify open source Java efforts
Bob discusses the pragmatic choice.
Peace breaks out
James is wondering if this will lead to a renewed JCP.
An Unexpected Pleasure
Mike discusses an end to the "Java Wars".
IBM and openJDK
Tim hints at the future of the Harmony project.
OpenJDK.Community += IBM
Dalibor hypes OpenJDK and welcomes IBM to the community.
IBM to Join OpenJDK
Mark highlights the area that IBM is likely to focus its contributions (Class Library) and looks forward to working with IBM engineers on OpenJDK and the JCP.

This is an exciting time to be working at IBM on the J9 JVM. I look forward to seeing how this new collaboration will enable a more rapid pace of change for the Java language and the runtime.

Update: Here is a more comprehensive round up of the reaction to the announcement from The Aquarium

IBM to Ship World's Fastest Microprocessor

2010-09-08T14:00:00.000-07:00

Fresh off the presses:

IBM to Ship World's Fastest Microprocessor

IBM (NYSE: IBM) today announced details of the world's fastest computer chip -- the microprocessor in a new version of the IBM mainframe that begins shipping to customers on Sept. 10.

You can read the rest here.

But I'd like to call out the footnotes:

(1)-Up to 60% increase in total system capacity for Data managed with DB2 and IMS. As measured by IBM Large System Performance Reference (LSPR) workloads using z/OS® 1.11

(2)-Up to a 66% improvement in Java performance with Linux on System z. The improvement on z196 compared to z10 was measured using a single Java application that focus on a variety of Java Runtime Environment (JRE) application functions typically used in both client and servers.

Interview Tip: Legible Handwriting

2010-09-04T05:24:00.000-07:00

As I mentioned in my previous posting, we're hiring.

Of course with that announcement comes the inevitable interviewing process. Wherein a couple of us will sit in a room and interview a candidate. Don't be shocked, we ask you to write code for us.

Well, here is my "tip of the day" or as we would say on twitter #protip: when writing code (or likely anything) on a whiteboard (or other surface) make sure the interviewer can read your handwriting!

If I can't read your code I won't be able to tell if you wrote it correctly. I'll simply assume you haven't (I know, I'm mean). But really, do you want the job or not? If you do, prove it to me. Don't waste my time writing illegible chicken scratch.

We're Hiring

2010-09-01T12:58:00.000-07:00

What Pete said.

And I guess I should welcome my colleague Peter Burka to the world of blogging. Although, I appear to be about a month late.

Updated IBM Java 7 Beta Available

2010-06-02T10:08:00.000-07:00

The second release of the IBM Java 7 Beta occurred earlier today. Now available for download are SDKs for Linux and AIX.

Try them out and let us know what you think.

Two Great dW articles on Java 7

2010-04-22T14:25:00.000-07:00

The first developerWorks article is on the current state of Java 7 from IBM's perspective.

A highlight being Trent Gray-Donald's podcast found mid-way in the article.

The second is a roundtable discussion on the state of Java.

IBM Java 7 Open Beta

2010-03-09T10:23:00.000-08:00

IBM has launched an Open beta program for IBM SDK for Java 7 on the Linux platform.

Please go and give it a try and give us your feedback.

Benchmark fibs

2009-12-11T12:08:00.000-08:00

Elisabeth Stahl over at "Benchmarking and systems performance" has a great post on Sun's latest press release announcing their SAP SD 2-tier benchmark results.

Sun is trying to look good comparing a 48 core 256 GB system to an 8 core 64 GB system.

Closures in Java 7, not the usual suspects

2009-11-25T12:06:00.000-08:00

Mark Reinhold reveals some motivation and initial details on the inclusion of closures in Java 7.

The key motivator: multicore processors. To exploit multicore, we need to write good parallel code. As Mark says:

Working with parallel arrays in Java, unfortunately, requires lots of boilerplate code to solve even simple problems.

Closures can eliminate that boilerplate.

The adopted proposal? Not any of the original three (BGGA, CICE, and FCM), but rather a new one.

Mark outlines his feature list for a “working programmer’s language.”

To support the principal use case of parallel programming we really only need two key features:

A literal syntax, for writing closures, and

Function types, so that closures are ﬁrst-class citizens in the type system.

To integrate closures with the rest of the language and the platform we need two additional features:

Closure conversion, so that a closure of appropriate type can be used where an object of a single-method interface or abstract class is required, and

Extension methods, so that closure-oriented bulk-data methods can be retroﬁtted onto existing libraries, and in particular the Collections Framework, without breaking compatibility.

A couple of other integration features worth considering are ﬁrst-class method references and the ability of function types to include exception parameters.

So it looks like Sun/Oracle will implement this new fourth proposal and submit it as a new JSR for inclusion in Java 7.

I'm happy to see closures back on the roadmap for Java 7 and will be very interested to see how this evolves.

Avoiding JNI pitfalls

2009-07-08T13:24:00.000-07:00

There is a great new article on developerWorks entitled "Best practices for using the Java Native Interface".

Here is the summary:

The Java™ Native Interface (JNI) is a standard Java API that enables Java code to integrate with code written in other programming languages. JNI can be a key element in your toolkit if you want to leverage existing code assets — for example, in a service-oriented architecture (SOA) or a cloud-based system. But when used without due care, JNI can quickly lead to poorly performing and unstable applications. This article identifies the top 10 JNI programming pitfalls, provides best practices for avoiding them, and introduces the tools available for implementing these practices.

I think my "favorite" pitfall is using the wrong JNIEnv. I've seen many a crash due to this error. Usually, this occurs because someone thought it was a good idea to cache the JNIEnv once and use the cached version. Problem is that the JNIEnv is thread specific and the cached one can end up being used from a thread other than the one the JNIEnv belongs to.

Caching of the JNIEnv is typically attempted when integrating legacy code into a Java application where you would need to change many function signatures to accommodate passing the JNIEnv down to the right function. Instead of caching the JNIEnv, cache the JavaVM; it is not thread specific and can be used to obtain the appropriate JNIEnv using the GetEnv() call.

Troubleshooting Resources

2009-06-12T08:29:00.001-07:00

I've gathered below some useful resources when trying to diagnose a Java problem when running IBM Java SDKs.

Troubleshooting InfoCenter

Guided debugging for Java

IBM Guided Activity Assistant

IBM® Guided Activity Assistant is a tool that guides you step-by-step through troubleshooting and configuration tasks. It helps you identify symptoms, collect diagnostic data, analyze the collected data, determine a root cause, and apply a solution to resolve the symptoms. If you cannot resolve the problem, IBM Guided Activity Assistant helps you export the troubleshooting case and send it to IBM so they can investigate the problem.

IBM Monitoring and Diagnostic Tools for Java

IBM provides tooling and documentation to assist in the understanding, monitoring, and problem diagnosis of applications and deployments running IBM Runtime Environments for Java.

Diagnosis documentation

This diagnosis information covers IBM-specific features of IBM's offerings. New or revised material is clearly shown by revision bars to the left of the changes and by red text in online versions.

developerWorks Forum: IBM Java Runtimes and SDKs

The IBM Java Technology Centre (JTC) produces development kits for IBM's most popular platforms. Technical Lead for IBM's Java on Unix Support Chris Bailey and other members of the IBM JTC team invite you to this discussion forum to share knowledge and ask questions about your experiences of using the various IBM runtimes and kits.

Hudson Plugin for Eclipse Project Set Files

2009-06-08T12:09:00.000-07:00

We track some of our source code configurations using Eclipse Project Set Files (PSFs). We also use Hudson for portions of our build infrastructure. Making them work together, so far, has required relying on some ant tasks that we have built to handle PSFs.

Not having PSFs fully integrated into Hudson has limited our ability to use some of the nicer features that Hudson has to offer (e.g., Poll SCM, with a PSF as a Source Code Management entry).

I've searched around the web and have been unable to find a Hudson plugin for PSFs.

So I intend to build one myself. I'll post some progress updates here to let you know how I am doing.

IBM Real Time Application Execution Optimizer for Java

2009-06-04T13:41:00.000-07:00

There is a new tool available on alphaWorks: IBM Real Time Application Execution Optimizer for Java

This tool will take a "compiled Java application" (AKA a set of jar files) and will optimize it in various ways.

From alphaWorks:

The tool provides the following functions:

Escape analysis of objects per method invocation
Control flow analysis that splits an application into archives according to thread accessibility
Control flow analysis that detects potential occurrences of real-time java runtime errors MemoryAccessError, IllegalAssignmentError, IllegalThreadStateException
Control flow analysis that determines entry points into an application
Addition of stackmaps to Java class files
Verification of Java class files
Auto-generation of classes that will load and initialize all other classes within the same archive
Specialized packaging of Java applications into deployable archives by packaging all referenced
Classes from a dual class path
Removal of unwanted attributes from Java class files

It also does some very nice things when optimizing for Real-time Java (JSR 1).

From alphaWorks:

The tool separates application classes into the following categories:

NoHeapRealtimeThread (NHRT) accessible
RealtimeThread accessible
regular thread accessible
inaccessible: classes not accessible to any thread.

Health Center (Tech Preview) Version 0.5

2009-05-29T12:56:00.000-07:00

IBM Monitoring and Diagnostic Tools for Java - Health Center v0.5 has recently been released.

I've had a hand helping get this product ready and feel proud to see it now available for use in the wild. Please download it, try it out, and give us your feedback.

IBM "Watson" takes on Jeopardy

2009-04-27T12:27:00.000-07:00

Another thing that makes working at IBM cool.

STDT Guinea Pig

2008-10-10T12:48:00.000-07:00

Being a member of the J9 Virtual Machine team I have had the opportunity to use Smalltalk in my day to day job. Now, I get to be a guinea pig for a new Smalltalk IDE called STDT (Smalltalk Development Tools).

My first impressions are quite positive. This is a real upgrade in terms of being able to integrate Smalltalk changesets and C changesets together. In the past if you submitted code that needed to ported to another stream or, heaven forbid, backed out due to build or test issues, backing out the Smalltalk portion using ENVY was onerous. Now with the Smalltalk source being mastered in CVS (or any of your other favorite SCMs) it is a piece of cake to pick up a changeset and manipulate it.

Beep! Beep!

2008-06-10T16:29:00.000-07:00

This is one reason why it is cool to work at IBM.

IBM Java 6 is GA

2007-12-08T05:41:00.001-08:00

IBM Java 6 was declared GA (Generally Available) on Friday December 7th. There was much celebration in the office for all the hard work that went into delivering this release. I am proud to have worked with the team that made this release possible.

You can download the results here: http://www-128.ibm.com/developerworks/java/jdk/index.html

Try it out and let us know what you think here: http://www.ibm.com/developerworks/forums/forum.jspa?forumID=367

Open TCK? Sadly, no.

2007-08-10T19:05:00.000-07:00

From Rich Green

For what it's worth, I completely understand why there is disagreement. There are fundamental principles and goals that separately define the GPL and Apache Software Licenses. Unlike the GPL, the Apache open source license does not require innovation to remain in the open. Java technology governed by the Apache license could be altered by any organization—commercial or non-profit—and rendered both incompatible and inaccessible to the community. The trust and value of “Write Once Run Anywhere” could not be upheld.

It's good to know that Rich understands that there is a disagreement and why. However, Rich implies that people or companies that take Apache Harmony source and alter it could then market that new source as Java. That is simply not the case. As that altered source would not have passed the TCK. He also implies that the Apache Harmony project would itself become incompatible. This is clearly ludicrous, since everyone would be able to see the current pass rate of Harmony against the TCK tests. Oh wait, no you can't, since Sun is not allowing Apache to have a license to run the TCK that doesn't include field of use restrictions.

The only way that any entity can distribute (for profit or not) a product and say that it is Java is if they have passed the TCK.

Removing the field of use restrictions being imposed on the Apache Harmony project would not allow entities outside the Harmony project who have modified Apache Harmony code to run the TCK under the license granted to Harmony project.

Any entity that wants to distribute a product and say that it is Java needs to secure their own TCK license.

What is Sun afraid of?

My Favorite Bug Report Summary of All Time!

2007-07-06T07:40:00.001-07:00

I was looking through some old bug reports and found one entitled:

"AlienHunter moves slowly once he digs a hole in the ground"

I immediately started to laugh as I thought to myself "...of course he's slow, he's tired because he just finished digging a hole."

Of course, the bug itself had more to do with calling System.gc() in a tight loop, but that's besides the point.

Project Zero

2007-07-02T12:12:00.001-07:00

Project Zero is an effort that involves a large cross section of the IBM software development community. There is also an external facing website that allows our customers to get involved, projectzero.org. If you're interested in web development I would recommend have a look around the Project Zero site.

Harmonicon

2007-05-08T13:13:00.000-07:00

Some more research has been done on top of the metronome garbage collector available in WebSphere Real Time.

Check it out here.

Death of Apache Harmony? I think not.

2007-04-10T13:11:00.000-07:00

Dave Gilbert is taking pleasure from watching Apache Harmony "die a slow and inevitable death".

There is only one reason Harmony would die a slow death. The lack of access to the JCK. The ASF has sent an open letter to Sun about this issue and I hope that it gets resolved quickly.

Dave insists that Sun's OpenJDK makes the Apache Harmony project redundant. To an uneducated observer this would seem to be true. Why bother trying to create an open source Java SE implementation when the original is now open source? ... The license, that's why.

The Harmony project will continue to exist and thrive simply because of the ASF license, with the caveat that the JCK must be made available.

So Dave's prediction of the death of Harmony could come to pass, but not due to lack of interest from the community, but rather by lack of availability of the JCK.

ACM Queue - Realtime Garbage Collection

2007-02-14T14:56:00.000-08:00

There is a great article here that explains realtime garbage collection. Well worth a read.

http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=454

IBM J2SE 5.0 SR3 Available

2006-10-11T14:08:00.000-07:00

IBM has made J2SE 5.0 SR3 available.

You can download various installs from developerWorks.

JAviator

2006-10-11T08:26:00.000-07:00

Here is a really cool video of a realtime application built on top of Metronome.

Metronome is the GC behind IBM's WebSphere Real Time Java environment. It allows you to write realtime Java applications without having to manage your own memory. In other words garbage collection is back and happy in realtime scenarios.

Metronome works for realtime because it guarantees pause times. Currently maximum pauses times are in the few hundred microseconds range.

IBM Java2 JREs

2006-10-04T17:51:00.000-07:00

If you own an IBM PC manufactured prior to April 1st 2005, then you don't have to use the Sun Hotspot VM. You can use IBM's Java2 JREs.

You can download them from here.

Update: for those that do not own an IBM PC that old. You can also download the IBM Development Package for Eclipse. Inside this zip file you will an IBM Java 5 SDK. It can be used on any machine (IBM on non-IBM) because it is packaged with the eclipse IDE.

IBM JDK for Harmony

2006-10-04T17:43:00.000-07:00

This is old news, but I thought I should mention it anyway.

If you'd like to try a J9 VM you can. Download the "IBM Development Package for Apache Harmony"

This download allows you to use a version of the IBM J9 VM with the open source Apache Harmony project. The Harmony project's purpose is to create a completely open source implementation of Java SE 5.0.

J9 Articles on developerWorks

2006-09-13T10:25:00.000-07:00

Here is a series of 5 articles on IBM Java 5.0 (implemented with the J9 virtual machine) on developerWorks.

Real Time Java

2006-09-12T12:48:00.000-07:00

Real Time Java is now available for commercial use from IBM, via
"IBM WebSphere Real Time V1.0".

See the press release here and the product page here.