JavaOne 2002: Zig’s Notes

As usual, Zig took a ton of notes, typing blind into a PowerBook with the backlighting turned off. High speed, low accuracy. New for this 2002, Zig is no longer cleaning up or fleshing out his notes. Cleaning up and fleshing out 8-10 hours of notes (one day’s worth) takes 4-6 hours. In an attempt to cut that time down to 2-3 hours per day, Zig is merely reformatting the notes into XHTML, and inserting a summary at the start of each session’s notes.

Monday

Keynote 1

Welcome to JavaOne 2002. This year’s conference will is focused on technical sessions, not CEOs. There is increasing energy moving Java forward. The Java Community Process works well to grow Java. Expect to see 10^14 things on the network. Java is the most popular programming language in the world. There is far more to the network than just desktops and servers: cellphones, light switches, and so on. The span of Java Personal Edition, J2ME, J2SE, and J2EE support a wide range of life in this network ecology.

Welcome

Java Momentum Increasing

Solaris, Java, XML, Sun Developer’s Network, Forte Tools, all under Pat’s control.

We keep Java and added into Java so that we an enable all of the applications that run on it. [ZZ: Is there any content in that statement?]

This year, JavaOne is focused on technical stuff, the developer. Less business focus.

7,600 attendees. All talking about new applications, salesforce automation, wireless, decision support.

Java Community Process

Operating Systems are commodity, the world we’re in today is a TCP/IP and JVM world.

Now with JVM we’re more about the ability to build new kinds of applications and box-based computing is just one part of the eCosystem.

Tomorrow we’ll talk about how to move the Java community process to the open source world.

An Internet of 10^14 Things

Experimenting with various Java devices, chips that go into paint for airplane fuselages.

Java Web Services pack, Java XML pack

Java is Popular

Java language now used by 56% developers, most popular programming language in the world.

Think Beyond the Desktop

The java organization is increasingly looking at clients in the aggregate: more than just desktops. We have specialized forms for the various client capacities: J2ME, J2SE, J2EE.

The commonality of the platforms gives you portability of your sills and technology

[ZZ: want a cheap portable probe? Some of those J2ME devices and aJile devices certainly could support promiscuous mode capture. The Sharp Zaurus PDA sells for $200 at the show, it’s linux + PDA. ($550 street) I’m tempted to buy one just to see if it’s got tcpdump and see if I can roll a quick & dirty Java capture wrapper.]

How can I get my work in the hands of millions of people? The Nokia TradePoint marketplace lets you show your stuff to all sorts of phone folks.

Java on Tiny Devices

JSR 172 extends XML parsing, RPC to the tiny little J2ME devices via two optional packages.

Project Monty

2 VMs increase performance 10x for CLDS and CDC. Conserves battery life, smaller footprint, delivers richer content and faster applications to mid-end mobile phones and PDAs.

J2SE 1.4/merlin

simultaneous release for Win32, Solaris, Linux (no Mac, though: I learned later in the Mac OS X session that 1.4 will be first available as a beta at WWDC in May. Maybe.)

Java for XP

There is an early access version for Java fo XP, integrates with IE6.0 and Navigator.

Call up Intel OEMs and make sure they’re including the latest Java VM in their built PCs.

J2EE

J2EE enables faster development, offers freedom of choice, simplifies business integration

Before J2EE, it was hard to imagine a consistent app server application without reengineering a lot of core technology. J2EE allows application folks to focus on their application, not th complex plumbing that server applications need.

It took only 4 months to get 13 compatible J2EE 1.3 products. There’s a lot of interest by developers who feel compelled to deliver compatible products.

Microsoft Bashing/Rebuttals

IBM and Oracle did a comparison, and their optimized code is 18-22x faster than Microsoft’s optimized version of the same Pet Store example. Bleah! :-P

[There were also several skits and other slams at Microsoft and .net. And yet they run their show monitors on Windows software. First clean your disk of Microsoft software, then slam them if you still think that much of Microsoft. Remember, the opposite of love is not hate: the opposite of love is indifference.]

More New Stuff

New: J2EE application verification kit (AVK): It’s a portability test environment based on J2EE 1.3.1

Java Web Services Developer Pack 1.0 in June 2002 to accelerate web services development. All the things you need, the Java XML pack, registry server, web application, tomcat, more. Go get it.

Keynote 2

Get quick definitions for many Java acronyms.

There's a lot of web services this year, so we'll talk about it a lot. But there's other stuff, too.

Web Services

We see J2SE as being the universal client. A lot of core XML support goes into J2SE.

SOAP: Simple Object Access Protocol

You can do more than request/response with SOAP. You can use it for async RPC and object invocation.

WSDL: Web Services Definition Language

It's for tools to use, not humans. Unlike CORBA IDL, human developers do NOT write WSDL. It's too rich, too arcane for humans. (We saw some WSDL in a demo, and it is indeed verbose, as most XML is.)

UDDI: Universal Description, Discover, Integration

Original focus of UDDI was very dynamic. It is now used more conservatively, companies set up UDDI registries of proven/trusted other businesses. More of a development time facility than used at runtime to discover new services.

ebXML

JAX-RPC

One of the reasons people use SOAP is interoperability between different languages. Otherwise you could just use Java RMI.

JAXP 1.1: Java API for XML Parsing

Coming soon: JAXP 1.2 adds W3C schema support: W3C schema is a far richer definition language than DTDs that lets you define your XML structure.

JAXR: Java API for XML Registries

JAXM: Java API for XML Messaging

JAXB: Java API for XML Binding

Important for easy access to complex XML, much simpler than using XML parse trees.

Security

JAAS Java Authentication and Authorization Services

Liberty Standard

Using Java to create national ID cards and enhance corporate data mining and unsolicited commercial email opportunities. But maybe that's just my opinion.

J2ME Clients of Web Standards

Need to investigate compact, efficient encoding of messages: verbose XML will blow out communication bandwidth of some devices.

Dumb clients: Browser Without Java Support

The create the lowest common denominator for a client class. They feed on XML or HTML or WAP.

GUI quality is sucky, and bandwidth consumption for presentation is inefficient, require connectivity and lots of it.

Smarter Clients: J2SE

J2SE as a client allows for smarts on the client, less connectivity required, less sucky UI.

Deploying J2SE

Java Web Start software: launches web-deployed apps fro a browser App is downloaded as needed and cached. Less dependency on browser.

The "consumer plug-in"

J2ME: Wireless

Unified emulator interface so you can pretend you're running on RIM, Motorola, Palm, other devices.

Forte, JBuilder, CodeWarrior, Visual Cafe all support developing for these devices.

J2ME: wired

automotive telematics, tv set tops, high-end PDAs, similar levels of processing power and memory, more than you typically see in wireless devices like phones

Most of these markets are still appearing, they lack the stronger definition of the phone market. The European Digital Broadcast process is one.

Connected Device Configuration CDC and Foundation Profile FP define embedded and GUI-less devices

The Littlest Client: Java Card

Secure authentication and personalization in wireless services (Project Liberty), ID badges, desktop login, chip-0enabled credit/debit cards

J2SE 1.4: Merlin

Tiger: J2SE 1.5

How the Leopard Got Its Spots: Design Principles of Jini

Describe why Jini is the way it is, so that more people can contribute to Jini without ruining Jini. Jini is designed to be very small and simple, with the hope that small and simple components can be reliable. Out of a collection of these small, simple, and reliable components, we can build larger, interesting network systems. We want our network to survive and evolve similar to how the phone system has stayed up for decades.

Jini is a community effort, more and more. There are major Jini developments going on outside of Sun. if the technology is going to hang together and retain coherency, we need to explain the design philosophy so folks outside the group can contribute.

The simpler you can make something, the more reliable you can make it. Reduce things to the smallest that you could. Have lots of little things interacting in well-defined ways. By being simple and having small,components, you can build a system that is reliable.

We want to build a network that long outlasts all the components that construct it. For example, the US phone system is a network that has existed and lived for decades, far longer than any of its original equipment.

We have to think in terms of the network being its own entity, above and distinct from the components that construct it.

Most of the things you would do with Java today you could do long ago with UCSD Pascal, but back then you were doing it on a PDP-11 and life was too short to wait for the results.

Networks are too large, too important to ever be shut down tin order to change. So the network must scale over time, last the way that the phone network has lasted.

We want to avoid any central point of failure or control. We want to give up control, assume that you do not have control. Let go.

We want change to be localized: coordination does not scale, so don't require coordination.

Minimal Infrastructure

Infrastructure must be minimal: get out of my way. Don't guess the future: you're wrong. Minimal = fewest assumptions about the future.

Jini: provide a place for services to advertise themselves, and then let clients find those services and use them.

Every service should do ONE thing well. Only one thing. Be small: it allows a service to be reliable. Combine multiple services for complex services.

Minimal Everything

Aggregations of small services have advantages: they don't all fail at once. They can deal with component failure. They can evolve, replacing pieces with new pieces. Find alternatives for failed pieces.

Why Java?

A single language: no need to map from IDL to implementation. No lowest-common denominator limits. All this IDL stuff introduce massive complexity. How will this data be interpreted in all of the possible language implementations? You end up with some minimal language definition.

portable bytecodes, dynamic language, secure with verification and sandboxes and security policies, so you can move code from one place on the network to another. You can safely inject part of a service into your own address space and not worry about how a service behaves, just focus on it performing its job.

Let us talk about an interface, what something does, not about how object talk over the wire.

Real Distributed Objects

Service object exists in multiple locations. A service hands its proxy to a client, so it now spans the network, exists in multiple l=places. How the proxy and service communicate are *entirely* an implementation detail. Some services don't even need to talk over the network.

We can have dumber clients than before. They don't ned to know about the services, other than their interfaces. Don't need to know how to talk over the wire. Don't need to know about wire formats.

Services can evolve over time, won't break clients as you change it. You don't have to get it right the first time.

Automatic Federation

You only need one rendezvous point (lookup service) to form a federation. You can actually have multiple lookup services.

This is even beyond DNS-style automation: you don't need somebody to push an updated DNS map. Things appear, add to the federation, go away, leave the federation.

People have better things to do than add things to naming services or make sure tht things in the service are really there. We do not need the network to be something that rules our life.

Type-Based Identification

This arose subtly over time. We no longer care about a service name: we just want "give be something that can so X". Whether service can do X or X2 or X3, irrelevant. You don't need another set of naming conventions on top of the programmatic interface, which you already have to know anyway to write your client code. Programs are good at understanding type systems, bad at names. People are good at names, poor at type systems. By extending a known interface, you can still let old clients use the old interface, and new clients can use the new interface. Your network can evolve piecemeal.

Time-Based Conventions

Two components have coordinated failure conventions and yet never have to communicate about it: it's part of the lease protocol.

A lease is there rather than a heartbeat is because the lease timeout may be dramatically different for different services. A transaction may want to be far more short-lived than a hard disk store.

It is not a system decision, but an application and installation decision. Shorter lease times means your list of services will be less out of date, but generate more lease renewal traffic.

Q&A

How do I find a printer? This requires a printer interface, we have to have a printer interface.

How do I protect yourself against rogue interfaces: Wednesday Afternoon: the davis project session, about how you do authentication and security in a world in which you can download code.

I get a "prime number generator", but it turns out that it's a prime number generator and a keystroke recorder. Wednesday 1:30 in this room (132). It is in fact a problem. So far, we're a small community, we can rely on hidden channels like beating people up if they misbehave. You have this problem in any system.

How do I use Jini for distributed business services other than cameras and printers? Don't think of Jini as just devices. For instance, the JRun guys have a J2EE clustering implemented as a Jini services.

Jini vs. Web Services? Web services are all defined in SOAP and XML and such. In Jini, all of that are irrelevant implementation details. You can implement your service using SOAP and XML and all that, but how are you going to know (other than it's slow)?

Why is Jini such a hard sell? It's so obviously a win. You're asking the wrong guy. I'll pay you the $20 later.

Jini is not a standards body. if some people define a printer interface, and then somebody else define s a better printer interface, if people adopt one interface over the other, the market decides.

Is there a QOS definition in Jini? No, not even part of the lease agreement. The lease just says how long a reference sticks around. QOS is a really hard concept, and I'm not really sure how you'd define it. You can look for services in terms of attributes, so if you define a QOS attribute, you can search buy that.

What is the largest Jini application? Come to the jinifest on Wednesday at the Argent hotel to see how people are using Jini. It's cool. Freddie Mac is using Jini and JavaSpaces to deal with half a trillion bucks in loans. One guy build a 0.6 Teraflop supercomputer using Jini and spare parts.

Jini and JXTA: what's the difference, Jini is java, JXTA is generic. I don't see the difference. The real difference: Jini is the tied to Java. period. End of discussion. Jini really works because you move objects around the network. JXTA, ad far as I understand it, is based on XML. It's basically a question of where you draw you homogeneity layer. JXTA drew it low, with XML . It's wire protocol definition. Jini drew it higher, at Java.

Jini Has Emerged: Some Successful Case Study Examples

Jini is used successfully in some large distributed systems. Raytheon is using Jini in ships for the Navy. Ericsson uses Jini for a nationwide defense system in Sweden. A tourism company in Italy uses Jini over the internet to connect heterogenous hotel, car rental, ticket, and other tourism-related companies. Ford is looking to Jini to provide a long-lived system in a car. Serveral references to a Quality of Service extension to Jini (Rio?).

Sun Professional Services

Distributed systems are way too complex. Hard to connect, locate services, administer system. Have to coordinate changes, roll change throughout system simultaneously, change tends to cause catastrophic failure: have to reboot the whole system to get it running again

The heterogeneity of devices, the nomadic, sporadic connection, all complicated things.

Dynamic self-forming systems allows for all this, makes much of that complexity go away. [ZZ: But not all: still have to deal with failures, initial service installation/startup.]

Need to provision services through a network using QOS attributes. Declare qualities (TCP, CPU, etc) and quantity (utilization, memory, etc) as attributes in an extensible model.

Applications that live on the network are provisionable through the network, not tied to specific compute resources.

Raytheon Naval

This effort forms the proposed foundation of the Navy's next generation of ships

lookup services allow the systems to self-assimilate: they find each other and pull the system together.

These platforms (ships, helicopters, etc) are *very* dynamic, the makeup of a force may change at any time. The environment is changing, dramatically, quickly.

The sensors on the ship transfer their information to the applications and services that need them.

All of the applications and services are monitored. If there are change in them, that's immediately realized by the network and it reconfigures as necessary to continue operating.

Ford

Cars live for 10 years or longer. We need a system that can be stable for 10 years. Jini allows for a network to live that long and evolve.

How can you stage an ownership experience that is rich and long-lasting when you have to plan the details years in advance?

The "carrier" telco/isp notion of quality of service metric is becoming critical to the service that begin and end at the vehicle. How do we design and engineer for zero unanticipated down time? How can we turn planned down-time into a pleasurable consumer experience?

T6: Some Italian Tourism Company

Offer high level complex services that are composed of basic tourism-related services provided by different organizations

Jini over the Internet: Problems, Solutions

Network latency uses Kalman filter algorithm. Gets accurate information out of inaccurate data. It supports estimations of past, present, and even future stats, not knowing the nature of the modeled system.

System components

An example: we added a Greek Ferry company's reservation system to the network with only 10 days of coding.

Ericsson Microwave

Java on Mac OS X: J2SE and Much More

Java on Mac OS X is great. JDK 1.4 will not be available for Mac OS X until WWDC 2002 (early beta) at the earliest.

[zz this session was pretty much content-free. It was not until Q&A when we pushed and pushed that we even got a mention of JDK 1.4 and a date. Other than that, nothing new to anyone who's programmed Java on Mac OS X for the last year.]

Java Reality Check Demo

ThinkFree Office

Get Amped

Stop by the booth and sign up for the Wednesday/Thursday Java Compatibility Lab: 15 dual-G4 computers and Apple engineers there to answer your questions. [ZZ: Sounds cool, except I don't actually have any questions now: Not until I spend a week implementing java plugins for EtherPeek]

Java on Mac OS X is Here

You can actually ship shrinkwarp applications in Java. It's here, it's real, it works.

The Swing API conforms to Aqua interface. Okay, not really. You probably need to tune your app so that it looks right on the Mac. The Java layout model is box-based, Aqua layout guidelines are more fixed-position based.

JDirect is an extremely efficient calling mechanism. [zz After speaking with an engineer, this isn't really accurate. JDirect is no better than JNI. All JDirect buys you is the ability to call existing native APIs without having to write JNI code. You still have the same marshalling/language boundary performance hit.

Java 2D API is accelerated, matches and often greatly exceeds default Java 2D performance.

Demo

Showed off some 2D and 3D graphics demo apps, written in Java, that used Java 2D and Open GL for blindingly fast rendering.

Q&A

Use the Software Update mechanism built into the system. JDK updates would be delivered to customers just like any other system update. [ZZ already this happened with the recent Java update.]

Prereleases of 1.4 will be on ADC sometime, we're working on having a first beta available at WWDC, but it'll be really prerelease, and I'm not making any promises today.

Tuesday

How to Build an Awesome Java-Enabled Client

How do you create an exciting GUI application using Java? You can use gratuitous graphic tricks like translucent images during drag and drop and fading windows in/out on open/close. Use resource bundles and relative text positioning to support internationalization. Use lazy or threaded view loading for faster performance.

Shiny new UI

Internationalization

the majority of potential customers do not speak your language, whatever your language is.

Java APIs solve generic issues, you still have to solve your application-specific ones.

J2SE solves geneeric issues like language, writing systemm, UI layout, currency, date formats, calendar, sorrting, text/work breaks.

ResourceBundle bundle = ResourceBundle.getBundle( "foo" ); JMenu editMenu = new JMenu( bundle.getString( "editMenu" );

Collator collator = Collator.getInstance(); Arrays.sort( productNames, collator );

Graphics Tips and Tricks

Lazily load parts of the applications. Show one window, spawn a thread to continue loading in the background.

Splash Screen: How?

Frame frame = new Frame(); MediaTracker mt = new MediaTracker( frame ); Toolkit toolkit = Toolkit.getDefautlToolkit(); URL url = getClass().getResource( "splash.gif" ); Image image = toolkit.getImage( url ); mt.addImage( image ); frame.setUndecorated( true ); frame.add( new ImageComponent( image ) ); frame.pack(); frame.show();

Lazily load JTabbedPane

for ( int i = 0; i < tabs.length, i++ ) { tabbedPane.add( tabs[], null ); } on tab change (ChangeListener) if ( tabbedPane.getComponentAt( index ) == null ) { tabbedPane.setComponentAt( index, myCreateComponentForTab( index ) ); }

Special Effects

Translucent drag: create a TransferHandler and install on JComponent (new 1.4 Drag and Drop class)

Use GlassPane to render image: replace GlassPane with customer Component that can render image. The GlassPane will be made visible and must not be opaque! Install DragSourceMotionListener to update the location and image and invoke repaint. Use Java 2D Composite and alpha channel for translucent draw.

To fade components in/out, create an image of the component, replace the component witha custome componaent that paints the buffered inmopage, and use a timer to invoke repaint with changing translucency.

RepaintManager rm = RepaintManager.currentManager( component ); Rectangle r = comp.getBounds(); Image buffer = comp.createImage( r.width, r.height ); Graphics g = buffer.getGraphics(); rm.setDoubleBufferingEnabled( false ); comp.setDoubleBuffered( false ); comp.paint( g ); comp.setDoubleBuffered( true ); rm. setDoubleBufferingEnabled( true );

Antialiasing.

High-Performance GUIs with JFC

You can dramatically improve the performance of your JFC-based GUI with a few tricks. setPrototypeItem() accelerates JList/JComboBox initialization. Be smart about how much you repaint(). Use reflection to avoid hundreds of little event adapter classes. Thread your UI. Use new image improvements.

setPrototypeItem()

setPrototypeItem() tells JList, JComboBox that "all the items will be no larger than this item. If you do this before you add items to the list/combo box, JList and JComboBox no longer have to do size calculations after each addition. Adding 0,000 item adds go from 300-700 milliseconds to 8 milliseconds.

Repaint as Little as Possible

Don't paint more than you need to: pay attendion to the clip. Example: a table: if it drew the whole matrix, it'd bring the CPU to its knees.

When something changes, just repaint that one cell, one row, the smallest affected region.

RepaintManager

RepaintManaager internals: HashTable maps from JComponenet to Retangle needing to be painted.

Unions dirty regions per JComponent. Coalesces/unions regions between children adn ancestors. Good implementation for typical/default case. You may want to extend/enhance for your own needs. For example, if you look at the clip and only draw the dirty regions but have to do hefty work per draw, it's better to get called once than twice. The Swing Connection will soon have an article on this.

Common actionListener pattern

addListener( new EventHandler() { public void eventHappened() { doSomthing(); } } );

This creates lots of little classes for event handlers (expensive! each class can weigh 4K memory)

ActionListner l = (ActionListener) EventHandler.create( ActionListner.class, this, "doSomething" ); button.addActionListener( l );

Swing vs. Threads

Swing is single-threaded. You should only call actions on components from the main event thread. setVisible(), show(), pack(), all these should only happen on event thread! Once you pack() or show() from main(), you should stop touching those components: they're now under the control of AWT event thread.

Safe from any thread: repaint(), invalidate(), revalidate(). Otherwise, use invokeLater() or invokeAndWait().

But do use threads! User actions should induce immediate feedback, let work occur in a background, give users a way to cancel lengthy actions.

Your models should send change notifications on the event thread! Changes usually induce GUI changes.

Never block the main event thread. When you block the event thread, nothing can be repainted. If the user drags the window off and on screen, big blank rectangle. Ugh.

Rendering Performance in JDK 1.4

VolatileImage

VolatileImage tells Java2D API to place imae in optimal memory for acceleration (aka put it on the graphics card if possible). But that memory may be volatile. Doesn't help much with translucency yet: we fall back to software in that case, and can even hurt if all your memory is off on a graphics card and we need to operate on it in software. First cut was to accelerate the common Swing graphic operations. If you think VolatileImage buffering is messing with you, you can turn it off with -Djavax.swing.volatileImageBufferEnabled=false

Toolkit Images

Toolkit.getImage() images are created in the depth of the screen [zz multiple displays: which one?]. They're system memory, but may implicitly accelerated. Good choice for sprites.

Images: When to Use Which One

BufferedImage: use whwen you really need an image of a specific format. Otherwise, avoid.

New I/O APIs for Java

java.nio.* gives us new I/O classes: channels are your basic I/O object, buffers hold bits. Memory-mapped I/O!

JDK 1.4 Features

Deferred Features That Missed the JDK 1.4 Boat

6 High Level Concepts That Make Up I/O in JDK 1.4

Channels

Async close and interrupt: you used to be able to close() an InputStream/OutputStream from thread B while thread A was blocked on a read()/write(): this was how to kill an async operation (and still is supported). But new exceptions make this more explicit. From thread B, you can interrupt thread A or close its channel if thread A is blocked on channel.read(). Only works for InterruptableChannel().

Old input/output stream system didn't handle extension, bidirectional, clean async. You can do connect, accept, etc depending on channel type.

ScatteringByteChannel: allows you to read into multiple buffers: for instance, if you care about the first N bytes as a header, and then the rest of the data is just data. GatheringByteChannel writes from multple buffers into a single channel.

Buffers

Support direct memory accccess and memory mapped files. Convienient access to primitive types: all except boolean.

ByteBuffer and MappedByteBuffer are probably most interesting, but there are buffer classes for IntBuffer, ShortBuffer, etc.

buffer data members

Relative gets and put respect the limit: don't read or write past the limit. They throw BufferUnder/OverflowException. Don't use the exceptions for flow control: they dramatically slow performance. use buffer.hasRemaining() instead.

A channel read/write is like a bulk get/put. Reads/writes up to buffer.getRemainingBytes().

Flip: sets limit to position, then sets position to zero. This one's counterintuitive, but it has uses. It lets you work on exactly what you just put/read something (it's the operation that most naturally goes between a read() and a write())

buf.clear(); for( ; ; ) { if ( in.read( buf ) < 0 && buf.position() == 0 ) { break; } buf.flip(); out.write( buf ); buf.compact(); // in case of partial write }

Duplicate Slices, Read-Only Views

ByteBuffer.duplicate(): same backing store, but position/limit/capacity are separate

asReadOnlyBuffer() creates a view into exactly the same bytes, but put operations throw. Immmutable data is cool.

Direct buffer: a buffer whose content (usually) resides in native memory, outside the normal garbage collected heap. It allows VM to avoid copying data during I/O. [zz would be perfect for the shared-memory ring buffer that EtherPeek for Mac OS X uses between the capture daemon and the GUI.] They are slightly more expensive to allocate/deallocate, so only use if you can explicitly prove with measurement that you're faster. 95% of the places where you might naively think a Direct buffer would speed things up, they really slow things down. You can create direct buffers from native code via JNI. This is how Java can talk directly to OpenGL.

Byte Order

Type-Specific Buffers: IntBuffer, LongBuffer, DoubleBuffer...

You can view a buffer asCharBuffer(), asIntBuffer(), and so on. Same data, but a separate position/limit/capacity. The view types are indexed in terms of their data unit size, not byte. They have bulk operations in terms of their types (float[], int[], etc). They are potentially more efficient.

File Channels

Note that absolute read/write completely ignore the position: multiple threads can read/write and not fight over the position.

Closing a file channel will also close the associated file stream. You can ignore the file stream once you create a file channel out of it.

mapped files: read only, read-write, and private (private allows writes, but doesn't propagate writes through to the underlying file system.)

Transfers

Transfers are a highly optimized kind of copy operations. Transfers can actually avoid copying data into memory: some channels support direct data transfer.

FileChannels: Locks

the invoking thead blocks until it gets the lock. You can lock the whole file, or specify a piece of a file. You can allow shared locks or exclusive access., You can test for a lock without blockinng if lock unavailable,. (tryLock())

Locks are tricky. Not all platforms support shared lcoks (pre-NT Windows doesn't). Your request for a shared lock might produce an exclusive lock. On some platforms, locks are advisory, not mandatory. Treat file locks as if they were advisory, and you won't have unpleasant surprises. Unix=advisory, Windows=mandatory. Do not count on locks being mandatory.

Tentative Futre Work

Formatting and scanning, leverage regular expressions. Really want something as compact as printf(). Scanning, something like scanf(), or at least a better regex-powered function.

Improved file system interface. More consistent actross platforms, more information when operations fail, better access to file attributes, pluggable interface to non-native filesystems.

More Effective Programming with Java

3 essays, excerpted from Effective Java Programming Language Guide. (Zig bought the book, borrow it from him or fork out $40 and buy a copy yourself.) Avoid needlessly creating objects: reuse objects where reasonable. Use defensive copying to protect yourself against callers changing your state out from under you. Use immutable objects to avoid worrying about changing state at all.

The book does not cover the GUI stuff, it covers java.lang and jaa.util, java.io.

Avoid Duplicate Objects

String s = String( "bob" ); // creates a new string every time it's called. String s = "bob"; // reuses the same object over and over

But don't be afraid to create objects as appropriate! Sometimes creating an object makes the code clearer, elegance, power.

Defensive Copying

Assume that calling code is actively trying to destroy your class' invariants and stability. This is usually just honest mistakes, but assume it anyway.

Watch out for mutable data members when you give out pointers to them. Watch out for directly using objects passed in as pointers: make a copy of them in case the calling code is hanging onto those pointers and might change the objects they point to.

Watch out for copy constructor vs. clone. Copy constructor slices the object, keeps around the part you care about. Dodges the chance that a subclass might change a value out from under you. You actually want to slice in this case: you don't want a derived class that might change out from under you. Your own data is your own data.

Real lesson: use immutable components! (Eliminates the need for defensive components.)

Immutable classes

Make sure that no methods can be overriden! Either make the class final, or make all your methods final (less optimal solution).

Immutable objects do NOT neet clone(), or copy constructor. Do NOT create clone() or copy constructor for immutable classes (String( String inOriginal ) was a mistake).

Immutable objects make greate Map/Set keys: if a key changes out from under a Map/Set, the Map/Set is corrupt.

Disadvantage: have to make a copy each time you do something. If you have common multistep operation, provide single functions to do all that work so that you can internally mess with mutable data, and then return an immutable result.

Alternative: provide a mutable companion class. StringBuffer is the mutable twin of String, and much faster for many operations like a + b + c + d.

When be immutable? Always, unless there's a good reason not to. All small data values should be immmutable. Date and Point were mistakes! Be mutable when they represent an enntitiy whose state changes, like a bank account, or iterators, collectios, threads, timers, etc.

If a class must be mutable, minimize its mutability. Make constructors fully construct/initialize. Don't leave objects partially initializied. Don't have "reinit()" methods.

The ease of working with a class is directly related to the number of states it can occupy. Immutable objects have a state count of 1. Look at how Thread is unreusable: linear progression of states, and once it's dead, it's dead.

Jini Service Life Cycle: Changes with Jini and JavaSpaces

How do you use Jini and Evolvable JavaSpaces to create a resilient system that can survive change to its components? After this session, I still have no clue. Judging from some of the Q&A at the end of the session, neither did much of the audience. Oh well.

After talking with their chief architect at the JiniFest, I have a better idea of what's going on. This session was really a 45-minute commercial, selling IntaMission IntaSpaces. They sell a product that solves this problem.

Refactoring is evolution's way of attempting to way to correct somthing that you got wrong.

Service lifecycle

Building a service may be difficult, but changing the same servie is harder. Defects bad be hard to find and isolate. Enhancements can often conflict with previous/existing work. Changes to persisten stats result in data migration. A service change regularly influences its syurrounding environment.

The system life cycle. The aggregation of services is born, the topology evolves, systems interconnect to form largert systems.

useful graph of each system's version across time. [zz no, I'm not drawing it into the notes.]

Changing one component breaks another, so we have to update the other. Then change another. It quickly becomes an interdependent mess.

JavaSpaces

Entries are the basis of JavaSpaces. They are the base entity. Everything that goes through a JavaSpace must be an Entry (interface net.jini.core.entry.Entry).

Spaces associate Entries. We find objects by associations. We make a template that looks like what we're looking for, then ask spaces to find something matching.

Refactoring in a JavaSpace

Use a JavaSpace to control factoring: space API is stable, Entry is volatile. Let the changes be in the Entry. Refactor at the Entry level.

Refactor not by refactoring the interfaces between the pieces of the system, but by refactoring the entries they pass around.

Incremental, refactoring programming is difficult in a deployed, living network/system.

Two levels of refactoring: the service level, where we aggregate new services and such, and there's the refactoring that we apply at the Entry level. Balance the demands of "taking the thing back into the hangar" vs. "walking out on the wing and pactching it up while we're in flight."

Example: adding a field to the "Contact" entry. Just change the object to add the field. How do we accommodate this change without taking the system down. The Calendar manager originally used the old object version. An upgradaded Contact manager starts spewing out new Contact objects. How will the Calendar react to the new Contact objects (or actually, two versions of Contact objects in one system). What if the calendar writes a new Contact back? Will it slice off the new field? You much design your system so that you retain ALL data of yoru entries.

Example: refactoring Van and Car into Vehicle, Van, and Car. This is a more interesting refactoring, since it mucks about with the inheritance hierarchy.

Evolvable Spaces: Refactoring While Live

An evolvable space requires no change to the JavaSpace-enabled interface. You add version management. Add online and dynamic partial upgrades. Forget the plumbing, solve the real problem. You've got to be able to build support for various versions.

How do Evolvable Spaces solve that problem? You need to deal with classes in different versions and dynamic class loaders TS1311: loading different versions of the same class into the same VM by using namespace management. Then work out how the information in the different versions map to each other. Build some sort of superset of that information, because you need to retain things. The resolve doing type-based lookups because you have different versions of the same class.

Get his book JavaSpaces in Practice (just finished). [zz I hope it's clearer and more helpful than his talk: I didn't quite get it.]

If you don't start out with evolvable spaces, you get into a chicken and egg problem: you can't change A without B, but B needs A....

Put an object in a space in form A, and pull it out in form B, and put it back in form B, and you lose A's info. That's why you need mapping abilities and different views of objects.

Creating a Product Line Architecture for Distributed Systems Using Jini

Some guy describes his Jini-based system. But he spends all his time describing domain-specific work and doesn't really touch on how he used Jini. He mentions that Jini helped, just not how.

SPACE

SPACE is a product line architecture to deliver TripWire's prodouct line: data integrity and monitoring tools.

They monitor the state of everyting: router congiruations, switch configuration, file sysem objects, registry entries, printer properties, anything that has state can be mniotored. We treat them all with Gather, Assert that it hasn't changed, and Respond if changed.

[zz Wow, maybe it's just me, but this talk is pretty content-free. It's just a guy describing his project's guts. Not noteworthy, mildly educational. Or maybe the last session of every day is content-free because Zig is consciousness-free.]

Techniques and Patterns

Securing communications between components: we use JSSE for SSL version 3, replaces the global RMI socket factory , use the JCE for bulk encryption, used standard java key-stores.

Wrapping jini transactions: enforce small lease times. Locally renewed leases using the lease renewal manager. Increases data intregrity. Consistently enforeced across framework. Short lease allows a failure to abort sooner.

Uniquely identified entries: allows all entries to be used in Distributed Collections. Consisten wayt ot retrieve entris from JavaSpaces component. Identity is an interface, allowing entries to be indentified in many different ways.

Distributed Composite Pattern: visitor iterates over identities, rather than entries themselves.

Wednesday

Jeri: Jini Extensible Remote Invocation

Jeri provides a pluggable implementation of Java RMI (remote method invocation). It works around known issues with the current RMI implementation. By being an open framework with replacable/pluggable pieces, Jeri lets you customize how it operates, so you can get RMI to work the way you require.

Jeri is an implementation of the RMI programming model. It's a culmination of all the RFE's for RMI over the years. These enhancements are either directly in JERI,or you can add them through customization hooks. You can use alternate transports, other than TCP. You can have dynamic proxies as stubs: you don't need to use rmic to generate the stubs. [zz That's cool: generating stubs sucks]

Motivation

We had always intended RMI to provide access to the architectural layers. But the architecture was in sun-specific implementations, so we wanted to provide non-sun-specifi implementation-specific ways to extend.

JERI

Exporter: uniform API to export/unexport a remote object. Implementation-specific info in constructor. Use Configuration to isolate source code from RMI implementation-specific APIs.

Configuration

The Configuration class allows deployment-time configuratbilty. No source code changes required. Don't have to recompile your application each time you change configurations. You obtain objects by name and type through Confgiguration.getEntry( String name, Class type )

Configuration.Provider is a implementation pluggable configuration. Default provider is ConfigurationFile, uses a bundle resource file in jar file.

Design Requirements

Support other connection types, including non-connection-oriented ones. Not just TCP.

Transport Layer

OutboundRequest has input, outputStream fields. Abstract I/O bytestreams from client to server and server to client (for response). On the server side, InboundRequest similar API.

ServerEndpoint has a listen() function. Specifies transport layer implementation at export time. Implementation determines request/response transport protocol.

Object Invocation Layer: ObjectEndpoint/RequestHandler

ObjectEndpoint can send/receive multiple objects. Responsible for remote object identification. Client side hangs onto object ID in the proxy, server-side has a table that maps object IDs to Dispatcher object. Protocol layer handles object identifiaction in request/responses. Handles distributed garbage collection, too (if enabled).

Remote Reference Layer: Dynamic Proxy + InvocationHandler/Dispatcher

Remote call comes in through InvocationHandler.invoke(). Encodes invocation and request stream (marshalling). Decodes results from response stream (unmarshalling). There is a standard UnicastInvocationHandler. You can chain them to dispatch to another handler (such as ActivatableInvocationHandler does).

Displatcher: decodes invocation from request stream (unmarshalling method/arguments), reflectively invokes the requested method, then encodes the return value/exception to the response stream (marshalling result).

RefFactory: used to specify remote reference layer implementation at export time. The InvocationHandler on the client and the Dispatcher on the server must be compatible: they have to understand each other's streams. RefFactory takes care of making sure you get a matching set. You can also chain RefFactories (such as ActivatableRefFactory does).

JeriExporter is the concrete class that implements Exporter for Jeri. Recommended: obtain one from a Configuration to defer choices to deployment time, rather than in code. Avoid hardcoded stuff.

HTTP-Based Transport Layer Implementation

Can chunk transfers. Has basic/digest authentication for HTTP proxy. Can be used with aggregate Endpoint implemention for a custom fallback strategy. Like the TCP implementation supports socket factory interfaces from java.rmi.server.

Secure Jeri

It's Jeri, but with support for security. Parallel architecture to Jeri, but with some Jeri types replaced with secure ones, and others extended. See TS 1171 Comprehensive Network Security for Jini if you want details.

Q&A

Exciting: you've ripped open RMI and allowed us to replace pieces that have caused us problems in the past. Even people not using Jini can benefit from this. Correct: it is a new RMI implementation, so it's a general thing, we just happen to be part of the Jini starter kit project Davis.

Programming Puzzlers

Two Java programming studs give 10 different example programs, each with a subtle unexpected behavior. These puzzles are all based on essays in Joshua Bloch's Effective Java Programming Language Guide.

1. Static functions are not dynamically dispatched

All static function calls are bound at compile time, not runtime. So you cannot override a static function. When you write code that "overrides" a static function, it isn't clear to some programmers which function gets called. The function associated with the actual compile-time class of the variable upon which you invoke the method is the winning function. It's silly and confusing to invoke static functions on variables: invoke them on class names instead.

2. If you override equals(), you must override hashCode()

Many classes (including HashSet) depend on hashCode(), not equals(), for object comparisons. You can lose objects in those collections without a hashCode() override.

// A good hashCode() recipe: public int hashCode() { return 31 * mField1.hashCode() + 17 * mField2.hashCode() ... + <prime-number> * mFieldN.hashCode(); // using prime numbers * fields so that "alpha bravo" // has a different hash value than "bravo alpha" }

3. Do not throw or return from a finally block

// What does this function return? public boolean foo() { try { return true; } finally { return false; } }

Since the finally executes last, it wins. The function returns false. But this kind of subtle code confuses mortals. And note that throwing or returning from a finally block aborts any exception processing from the try block. You just squelched an exception, bub. Evil, evil, evil.

4. int1 - int2 can overflow

This code fails if a=2,000,000,000 and b=-2,000,000,000. The result would be 4,000,000,000, which exceeds int's capacity (remember: int is signed).

Remember that just about any binary operation on two finite-capacity numbers can produce a result that exceeds that capacity. ints are not integers! Either use a larger-capacity variable for the result, or rewrite your code to avoid the overflow.

This also comes down to "don't write clever code": your clever code is probably wrong. Yes, the standard C/C++ sort code does the above.

5. 'h' + 'a' produces 169, not "ab"

Most people do System.out.println( 'a' + 'b' ) and get unexpected results (an integer value). Use System.out.println( "" + 'a' + 'b' ) if one of your operands might not be a string.

6. ambiguous code sucks

// what does this code print? class Foo { public Foo( Object o ) { print "Object"; } public Foo( double[] ) { print "double[]"; } public static void main( String[] args ) { Foo foo( null ); } }

Because an array of anything is actually a subclass of Object, and because ambiguous resolution will choose the "most specific" overload, the above code prints out "double[]". Will you remember than 3 weeks from now? Will the summer intern maintenance engineer know this? Disambiguate with casts "Foo foo( (Object) null );", or better yet, avoid this nonsense if possible with descriptive and different function names (no, this doesn't help for constructors).

7. bytes and shorts are promoted to ints

// this code prints nothing byte b = (byte) 0x90; if ( b == 0x90 ) System.out.println( "yep" );

The second 0x90 is promoted to an int, 0x000000090, or 144. 144 is larger than a (signed, remember?) byte can hold, so the comparison will never match. Either cast everybody to bytes (or shorts), or promote everybody to int and bitmask away the stuff you don't need.

8. float and double never hold the value you think they do

9. You cannot override data members

// this code will not compile class Base { public String mName = "Base"; } class Derived { private String mName = "Derived"; } System.out.println( new Derived().mName ); <-- compiler error: Derived.mName not accessible

There is no vtable dispatch for data members. Use functions (getName()) instead if you want dynamic dispatch.

10. Integer.MAX_VALUE really is

// infinite loop! for ( int i = 0; i <= Integer.MAX_VALUE; ++i ) { System.out.println( i ); }

In the above code, i can never be more than Integer.MAX_VALUE. When it's equal to MAX_VALUE, and you increment it, it overflows to Integer.MIN_VALUE. Watch those overflow conditions!

Comprehensive Network Security for Jini

This talk gives more details on secure Jini, under development as part of the Davis project. Secure Jini replaces some classes in Jeri to provide security. Secure Jini ties in with J2SE security and JAAS for authentication. You can trust that the server you're calling is really who you think it is, that your calls are getting there and back again without being tampered or eavesdropped.

"Jini is great, but what about security?" We want basic network security for remote calls. Mutual authentication so each other knows who's talking to. Want integrity so messages can't be tampered with, and confidentially so no eavesdroppers.

Non-uniform requirements. Different clients, servers, methods, data all could have different requirements. Just because Bob trusts Steve doesn't mean I trust Steve. There are so many ideas, schemes, we want to allow all of them.

Overall approach. Define a constraint system to express network security needs. Tie to JAAS Subjects. Provide a javaa RMI API model impelemntation that supports network security. Provide menchanimsm to verity trust in proxies.

Secure Remote Calls: You have to prepare the proxy: verify that you can trust the proxy, attach security constraints to the proxy, grant permissions to the proxy.

You can also have secirty preferences where you'd like to have X, but can live without it. "Prefer to be anonymous, but will authenticate if I have to."

Constraints

Security constraints: "don't care" = absences of constraints. Otherwise, all you can do is subtract rights. You can specify minimum constraints. You can aggregate constraints. You can apply constraints per method, so that different methods, such as authenticated writes, but anonymous reads. How do we specify constraints? Server specifies when it exports the remote object. Clients specify by attaching directly to the proxy, another is set a contextual constraint through Security.doConstrained( action, constraints ).

Calls fail if security constraints cannot be satisfied. Constriang not supported, conflicting constraints, client or proxy has insufficient permissions, insufficient credentials to authenticate, and so on.

JAAS

JAAS subject is a container of < principals (who you are), public credentials (public key), private credentials (private key) >

Use JAAS to log in, then you get a JAAS Subject object, and then you use that Subject for the rest of actions. Subject.doAs( client, action )

Some access control typically precedes argument unmarshalling to minimize denial of service attacks.

Object integrity: good data is useless without good code. Data integrity is handled by transport layer, code integrity requires integeirity protecting codebases. Code is downloaded out-of-band (lazy on-deman code downloading). HTTPS provides integrity of downloaded code. HTTPMD (message-digest) developed for Davis for downloading jar files. The URL contians the message digest checksum, so if it doesn't match, you know it didn't make it. There are other schemes possible, if you want.

Secure Jeri provides a pluggable secure remote invocation layer. [zz Jeri itself is quite pluggable, so this pluggability might just be a side effect of Jeri.]

Proxy Trust Verification

Problem: downloaded code. What prevents a proxy from lying to us? Can you trust the proxy to really be a proxy for your bank? Is it really just "proxy for bank" or is it really "proxy for bank, and keystroke recorder"? You know whoat the server should be at the other end of the proxy. We don't want to have to know the codebase, signers, orprotocols that the proxy/server use. We want to minimize client preconfiguration.

Easy trust case: all local code

Can verify proxy and check a transitive closurre of the proxy's classes and make sure they're all local, trusted classes. Tedius, but technically possible.

Hard case: downloaded code

Ask the server if it trusts the proxy. If we trust the server, and the server trusts the proxy, we can trust the proxy. Oh wait, we can't ask the server without going through the proxy, and the proxy could lie. So we cheat. We ask the proxy for a "boot[strap] proxy", which is all local non-downloaded code. Since we can verify and trust the local code, we can verify/trust the boot proxy. Then use that (now trusted) boot proxy to ask the server for a verifier object (can trust the server, object integrity means that the verifier from the server got here and we can trust the verifier). If the verifier verifies the proxy, we can trust the proxy.

Dynamic Permission Grants

Client-side authentication. We have to grant the proxy some kinds of permissions, such as to authenticate as the client when talking to the server. So we provide (indirect) access to our private credentials (assuming we trust the proxy). We don't know who to grant this to in advance. These are implementation details of the service. So we have to wait until we have the proxy before we can decide to grant permissions.

We require a java.security.Policy provider that can support this, so we provide one (the one in the standard JDK does not).

Deployment-time configuration

We want to make these choices at deployment time without having to change server and client code. The Configuration class does this. Default Configuration class loads from a resource bundle file.

Status

Methodical Design by Contract Using Assertions and Exceptions in Java

Design by Contract is a (fairly popular) coding methodology where you state preconditions, postconditions, and class invariants for each method. Tools or language features test these conditions before and after each method invocation, and spew errors when a condition is violated. JDK 1.4 adds "assert" that helps with precondition testing. There are competing third-party tools to offer richer support for postcondition and class invariants. The author just published Design by Contract, by Example from InferData.

6 Principles of Design by Contract

What is Design by Contract?

/* @require !(o == null ) @ensure !contains( o ) */ public void remove( Object o );

Contract is two-way: caller must satisfy precondition, implementation must satisfy postcondition.

In the right environment, you can switch on runtime support (and switch off, if you like) that tells you who violated the contract.

iContract is a free tool at www.reliable-systems.com that parses comments for conditions.

Principle 4: Postconditions for every command

This is really the heart of design by contract, and the hardest principal to grok.

Principle 5: add preconditions where appropriate

Prin 6: Invariants

These apply before and after ALL method invocations (although it's OK if they're violated during an invocation).

You can turn checking on and off by class. While developing class X, leave all checking on for X. When working on class Y that uses X, turn off X except for preconditions, leave X's preconditions on to catch problems in Y where it incorrectly calls X.

You can build a conceptual model of a class that has nothing to do with the actual model (such as a Vector as the conceptual model of a Queue). The conceptual model may have horrible performance, but if it simplifies the conceptual model well, you can write richer preconditions and postconditions in terms of that conceptual model much easier. Performance is acceptable, since it's only while testing/developing that test.

Design by Contract and Exceptions

People using exceptions probably have been using some art of design by contract without knowing it. If you ever threw due to bogus inputs, you're checking preconditions.

How to best start using contracts? Full contracts involve preconditions, postconditions, and invariants.

Working just with preconditions yield immediate benefits (Java libraries are full of examples). You don't need anyh special tools for this.

Two different uses of exceptions as precondition checks

throw a RuntimeException/RuntimeException subclass (dodge that annoying CheckedException try/catch noise)

Case 1 - exception thrown implies bug in calling code

public class PreconditionException extends RuntimeException { ... } public void remove() { if ( !( size() >= 1 ) throw new PreconditionException( "Queue:remove:size() >= 1 " ); ... }

Case 2 : exception implies an expected error condition, not a bug

public class QueueEmptySignal extends RuntimeEception { } public int size(); public void remove() { if ( !( size() >= 1 ) ) throw new QueueEmptySignal(); ... } // calling code while( more Elements ) { try { q.remove() } catch( QueueEmptySignal s ) { moreElements = false; } }

Here, you're using try/catch to test the precondition on remove. You expect the exception as part of flow control. Benefits: you're down to one call, not two. You dodge race conditions.

[zz Note that we've been repeated begged by Sun and JVM engineers to avoid doing this. Exceptions kill performance. They nuke branch predition, blow out pipelines, instruction caches, everything you can do to make a machine slow, they do. Just don't do it.]

You need tool support for proper contracts: you can turn it on and off, precondition in class Y, nothing in Y, post in Z, depending on where you're focusing.

Q&A

Isn't there overhead in using exceptions as flow control? Yes, it's a fair argument. In language where exceptions hurt, use integer return codes and check return codes.

Preconditions/postconditions for concurrency constraints? Contracts best worked on sequential code. For concurrent code, some of your correctness preconditions become "wait" conditions.

Precondition exceptions vs. IllegalArgument exception? This talk doesn't get into that details.

Can you compile out the JDK 1.4 assert: apparently yes, and there's no performance impact.

What about "nothing else changed" in the queue rules? I skipped over that. it's in the book.

JUnit and test-first? The contracts become test oracle/documentation that helps you write the tests.

By having the contracts in documentation, we have competing vendors working on giving you contract tools: this is a good thing. Even with assert, you still need the tools, they still give you more than assert. Assert cannot do postconditions, you lose access to pre-execution values.

Expo Floor and JiniFest

Seen at the show: Men's room urinal screens that read "Java: it runs right through you"

Also seen at the show: at least a dozen TiBooks in the hands of attendees, all running Mac OS X.

Also also seen at the show: about 1/2 of the laptop computers here are running linux. About half of the demo computers in the expo and up on stage are also running unix: either linux, solaris, or Mac OS X. All linux installations appear to be Red Hat: at least they use the same default window manager that Red Hat installs.

Sitrtka JProbe 4.0: might want to check it out. Zig has a demo disk. http://www.sitraka.com

Forte for Java, Community Edition: Sun's IDE, looks like your standard IDE. Zig has a demo CD.

IntaMission IntaSpaces: a JavaSpaces implementation that makes it trivial to support multiple versions of a class/service within a single Jini (well, JavaSpace) system. Single objects can be used simultaneously by clients/servers of multiple versions. Very cool, should provide for very stable, long-lived networks. http://www.intamission.com

GigaSpaces: a JavaSpaces implementation that provides replication and scaling. But more interesting is their installer: they install JavaSpaces, Jini, the rmid stuff, the http server, a JVM, security policies, *everything* you need to run their system. In a single one-button ZeroG installer. The installed product is a Win32 executable. http://www.gigaspaces.com

JStamp: A tiny real-time Java module. They produce hardware with a small amount of memory and a network interface. They could *not* be easily adapted to disk storage, though, nor to large amounts of RAM, so these are not really good for packet capture. They're targeted lower. http://www.jstamp.com

There were a few other companies selling tiny hardware boards with aJile Java processors on them. I bet we could look around and find somebody that could cobble together an aJile-based, J2SE or J2ME agent box with a Jini service.

Motorola Labs: a guy wrote a Jini system for network and system management with distributed agents. His Jini "agents" are SNMP monitors. You install the "pod" software on computers near what you want to monitor. The pod software fires up, finds a Jini lookup service, and registers itself. Then other Jini services notice the new (empty) pod, and tell the pod to download an agent. This experimental system has "smart proxies": the Jini proxy objects contain all of the smarts, there isn't really any server. So once the pod downloads the agent "client" and starts it, the agent starts monitoring SNMP devices. The system also contains trigger-action rule objects, which can tell an agent to move to a different pod (if the current pod is too slow), split into two pods, and so on. Dynamic load balancing.

There is a sourceforge open source project Jini-Tools that is looking to record community knowledge and make it easier for Jini developers to get their jobs done. They are looking for contributions of knowledge, code, energy. If we solve the installation/startup problem, they'd love to have it open sourced. Considering how much EtherPeek for Mac OS X benefits from darwin, bpf, libpcap, cppuint, and log4cpp, it's only fair that we contribute back to the community. http://sourceforge.net/projects/jini-tools/

ExpressChart: a charting component similar to the controls EtherPeek uses for its pie, column charts. Could save us a chunk of time. Probably a jillion similar products, just thumb through a Java Developers Journal and look at the ads. http://www.quadbase.com

Loox Software: a truly amazing network/graph layout library. Does a fantastic job of laying out nodes in a graph, keeping them out of each other's way, including labels. If we do a peer map in Java, we really should check out their stuff. Could save weeks of engineering time. http://www.loox.com

PsiNaptic: one of many hardware vendors selling small hardware with embedded Java. They have a custom engineering capacity, could help us develop a standalone network probe box. http://www.psinaptic.com

The Handout: Interesting URLs

There was a handout at the JiniFest with project summaries and URLs. Here it is typed into the notes, since I know I will lose the handout within 72 hours.

Benetech

Benetech applies tecnology to social needs in areas such as disability, human rights, and literacy. Benetech will be showing a prototype of accessible wireless devices for people with disabilities. This demo is enabled by Jini technology along with Service UI API.

Freddie Mac, Financial Engineering Group

A leader in mortgage banking will be demonstrating distributed financial applications powered by Jini and JavaSpaces technology.

GigaSpaces, Inc.

The GigaSpaces Platform is a commercial implementation of the JavaSpaces specification with many powerful enhancements, and enables collaboration, sharing, and exchanging of information. It is focused on enterprise distributed applications, mobile, and web services.

IntaMission

IntaMission's software revolutionizes the way large corporations deliver applications and services in volatile and complex environments. IntaSpaces is an evolvable commercial-strength implementation of the JavaSpaces specification.

Karora Technologies

Karora software products and services specialize in business integration for mid-size enterprises. They have component-based integration solutions that linke business processes, enterprise legacy applications, and databases within or across enterprises. These are powered by Jini and JavaSpaces technology.

Motorola Laboratories

Jini technology enabled services to the cell phone. Leveraging the Surrogate architecture and Service UI API to overcome limitations of small devices, Motorola Labs demonstrates wireless access to Jini technology services.

Procoma

Procoma’s Chameleon/XML (CXML) product is an enterprise scale system for transforming XML into business documents and other media. CXML employs Jini technology to enable scalability, resilience, and interprocess communication. Through Jini, nodes can be added to a cluster and automatically come on line and start sharing workload.

PsiNaptic, Inc.

PsiNaptic’s JMatos software extends the power of Jini technology to resource-constrained Java devices. With a memory footprint of less than 100KB, JMatos is ideal for J2ME-based mobile phones or PDAs, automobile subsystems, and resoure constrained medical devices.

Sun Microsystems, Inc.

The Jini Technology Starter Kit, version 1.2, is Sun’s implementation of the Jini specifications contributed to the Jini community.

Tripwire, Inc.

Tripwire is a data integrity assurance company which has employed Jini technology in ints products to create an enterprise architecture for performing data integrity tasks on heterogenous objects over the enterprise.

Valaran Corporation

Valaran’s V-aerial product suite is a distributed business infrastructure that unites design, implementation, and measurement of business processes. With V-aerial, new and existing applications, network elements and various devices can be integrated effortlessly, creating a common and evolvable infrastructure platform across the entire enterprise. V-aerial is able to address any integration challenge through the unique use of Jini techhnology.

Cyberspace

A framework that helps people (not software) find and use Jini services on the network. The project proposes a simple space metaphor by which users could relate to and use the services offered by the emerging meta-computer. In the metaphor established by the Place API, all services are represented by objects. Objects are organized into places. To do something on the meta-computer, uesrs go to a place and use an object. This ‘objects in places’ metaphor encourages users to think of the meta-computer as a space.

Clusters

The Clusters project aims to specify and develop tools for constructing large-scale computational cluster and grids with Jini. These tools center around a set of Java APIs, and a reference implementation of those APIs, that facilitate reuse of cluster-related functionality in a distributed system. The API offers dynamic discovery and incorproporation of nodes into a cluster, resource monitoring, failure detection, load balancing, and high availability (replication). An interesting aspect of the project is the development of migrating Java RMI remote objects: Such objects dynamically move between cluster nodes at runtime, carrying both their data and code, thus automating load distribution and facilitating replication of a cluster.

Davis

The Davis project is an effort by the Jini technology project team at Sun Microsystems to address the need for a Jini technology security architecture, as well as to contribute other improvements to Jini technology. The Davis project will ultimately result in a Jini Technology Starter Kit release from Sun. In the short run, the Davis project has produced the Overture release, currently available for download, which includes the basic security architecture needed to support further efforts.

NetBeans IDE plugin

NetBeans is an extensible, open source IDE. This project adds an extension to the NetBeans IDE to aid the development of Jini services. The NetBeans Jini module features service templates, a djinn browser, and a client code generator. It seamlessly cooperates with NetBeans RMI module.

Out-of-Box

We are trying to improve the out of the box experience, working on nicer installation, startup programs, and a show (demo) that is a good place to experiment.

Rio

Rio provides a simple and flexible approacch for a dynamic service delivery architecture based on the capability to provision services through the network using qualitative and quantitative QoS attributes.

ServiceUI

The goal of this project is to decide upon standard ways to attach any kind of user interface text-based, graphical, voice, etc. to serivces.

Surrogate

There are software and devices that can not directly interact with a Jini network. The goals of the Surrogate prroject are to investigate the use of third-party computational resource to “Jini enable” these devices and to define a standard way for doing this.

Thursday

Managing Network Elements with Evolvable JavaSpaces and Distributed Intelligent Agent

WorldCom, a big telecommunications company, is about to conduct a field trial using Jini and Evolvable JavaSpaces to manage their network. They have a network of many duplicate “pod” Jini services, which in turn download other Jini services that monitor and manage network devices near them on the network. Additional “angel” services monitor services, move them to other pods if a pod slows down, split monitoring burdens across multiple services and/or multiple pods to balance load, restart dead services in new pods. The entire system is resilient: huge portions of the system can fail, even simultaneously, and yet the system continues on.

Examples from the NewWave DIA

Telecommunications: We’re living ina world of decreasing revenues. We’re going to talk about ways in which the telecommunications world is adapting to overcome the erosion of revenues, so that we canmove into new produce spaces and endorese new technologies. And most important: recover the cost. We learn more and more that the amount of telecom revenue that goes into softwre is exttraordinary. The actual network only costs agout 15% of the cost to a service provider. The vast majority of the remaining cost is software, procuring the software, maintaining the softare.

How are we providing cost recovery? My job is to seed innovative groups in companies so that they can produce things that make my job easier.

Network things were pretty stupid things compared with modern computers. We haven’t evolved the control levels of these devices as much as computers.

Outrageous Prediction

Current Jini technology will so significantly reshape the development and delivery of large applicaiton systems, that in a decade virtual all networks will be delivering applications via Jini services.

WorldCom’s NewWave OSS

Started in 2000 using Jini. Java + Jini + Enterprise Extensions + Policy + SmartElement is how they did it. Highly distributed networks of Sun Netra class servers powered it. They improved efficiency, process re-engineering, faster time to market, globally scaled services, support, and the network survives. Whole sections of the app server network can fail at once or intermittently, but the netwokr survives. You never need a dedicated backup running on the network.

You see all these smart, tiny devices with Jini on them, and this huge network/telecom systems talking totally idiotic control protocols. Huh?

NewWave is a network aware enterprise-service-oriented architecture and technology. It’s a unifying layer on top of all these different services, and business components use it to talk to the services.

Jini gives us code mobility. Very powerful. Moves configuration off the box. Provides survivability. Its simple network service model adapts to plug and work networks, delivers GUI apps to clients. It gives us a new and different mechanism to acheive business model. Service goes to customer, offers service upsell to customer, customer agrees, service provisions the service to that customer. Literally quicker than it takes to describe it.

NewWave is a federated hierarchical system where each Jini domain is a child of another Jini domain. The lookup services are hooked together. A container is a VM that registers with Jini and provides some basic services. Remotely deployable. The infrastructre is self-managing and keeps the system running. There are agents (services) that watch other services to keep things running, called angels

They buy a stack of Netra-class Sun boxen, throw container services on each CPU (yes, multiple server processes on each CPU), and the lookup server knows about them.

Self-healing proxy reference

Client-side failover. If the proxy loses a connection to a service, it’ll go back to the lookup service, find a reference to the service, and reattach. Server-side rehoming: angel service monitors a service, discovers that the service goes away, it relaunches/rehomes a server in that container or another one.

Looking at JavaSpaces with many many spaces, as objects go into one space, they’ll move up as necessary in a hierarchy of spaces.

Jini domain: has an archangel service manager, each domain has a lookup service, archangel service manager, oodles of container processes. One top-level domain can spawn a hierarchy of domains, each domain has a template of appropriate services to put in that domain.

JavaSpaces

JavaSpaces: a shared repository for objects. A space itself is a Jini service. JavaSpaces have a very simple API. JavaSpaces span services. Client A and Client B both have their own imported proxy, flowing objects between themselves and the JavaSpace.

The JavaSpace server has to deal with multipl clients, serving each of them fairly. It has an indexing and storage syssytem to map matching attributes to the appropriate object. It optionally has a persistent store. It has a connection manager to talk to the clients. Spaces do not have to have persistent support: transient spaces do not have persistent storage. In the case of failure, persistent storage survives, transient does not. You move Entries thorugh the JavaSpace, Entry is the basic type of objects in JavaSpaces.

// NetworkElement version 1 class NetworkElement implements Entry { public Integer id; public String state; ...methods... }

What happens when information models change, but you can’t bounce the whole system to upgrade everybody all at once?

// NetworkElement version 2 class NetworkElement implements Entry { public Integer id; public String state; public String location; // new field! ...methods... }

IntaMission moved the problem to the space: service 1 that uses the fist version of NetworkElement, and service 2 uses version 2 of NetworkElement. Evolvable JavaSpaces allow that.

Aggregated JavaSpace model

Network events flow into bottom spaces, there are many spaces and services in these spaces. We’re thinking of lots of transient spaces with lots of (millions/billions per day) of transient objects going through these spaces. Then some services look at these events/objects, and send higher-level events up the stack to a smaller set of spaces. You flow up ta pyramid/hierarchy of spaces, connected by services to flow objects around.

NewWave Distrubutive Intelligent Agent

There’s something neat with Jini and JavaSpaces: they’re a good thing to try. It’s worth taking the risk to see if they help you. They worked for us. Take the risk: “A ship is safe in port, but that’s not what it’s built for.”

We have a network of several hundred thousand elements in a few thousand customers. How do you manage that? It’s a big problem for us. It has business value: you can save dollars and have better customer support. Jini has good and bad things. We want a machine-to-machine solution, this is a machine-to-machine problem. We can distribute agents out to the colocated network equipment. Bad thing: other attempts have faild. Traditional IT people do not believe that you can have distributed system that work.

It’s a Jini service. It has little services inside it. It’s like a shell. It’s this shell we distribute near the elements (the network stuff they manage). Inside each shell is a JavaSpace and a bunch of small services. Thee services are small, have clear and small function. We don’t look for JavaSpaces to hold data for a long time. The JavaSpaces are there for interservice communication within the shell. We sync the DIA (the shell) down near the elements they manage. This minimized traffic across the network. We don’t want an alarm in Hong King to show up in the United States.

Who reads or takes what from the space? Objects are ordered by channel messaging: a channel between two services is created, that gives an order and control over services. If a server isn’t keeping up, you can fire up another service to dynamically load balance.

We started two projects in parallel: this Jini approach, and a CORBA project. The CORBA prototype failed miserably, and so we hardened the Jini project. This year we’re doing a full field trial. We truly believe it will be successful, and that next year we will deploy this around the world.

Jini is getting more mature. Jini is pretty well understood by Jini people now. You can sit around have converstaions about some of the things you really can do with Jini today, and these conversations are completely unbelievable: nobody could fathom the amazing things you can really do with Jini. 5 years ago nobody would have believed it. Today, nobody who doesn't use Jini will believe you. So now we have to move on to the next challenge: convincing people doing older-style systems and showing them that they can interoperate with Jini and benefit from Jini.

We used all sorts of agent systems, and none of them worked out. Even J2SE and J2EE systems. None of them mapped well to the problem. Jini has been remarkably easy to evolve to our requirements.

The deployment model for your services is fundamental to your success. You need to be able to push services right next to where you need them. You need to be able to remotely fire up services.

Everything in telecom is really long-lived. Nothing ever seems to go away. So evolvable JavaSpaces are critical to being able to support things for a really really long time.

Q&A

You talked a lot about agents. What’s the relationship between the work you’ve done today and the worrk you’re still carrying on? What about the JSR for agent interfaces? I’m not using that JSR work, so I’ve no clue. We worked first with mobile agent IBM aglet model, we learned that you didn’t need to have the itinerary inside the agent itself, in fact it was better to deploy the itinerary externally. The agent can be cloned and moved around much easier. We made the mistake thinking agents are self-contained entities: they are better as components of entities. There are a few places where I might try true self-contained agents, but that’s research. For today, what we can do with remotely deployable services is so poowerful that we really don’t need to go beyond that. I have what I need.

Is it possible to find out more details on the architecture and particulars of the hierarchy of JavaSpaces, howyou create a new one, dynamic?, How to you communiate between different Spaces, and so on. We’re working on making the internal details of this a bit more public. We have to go through a lot of “intellectual property” rigamarole, we’re working on it. Big telco company, ya know. The concept we have is that the DIAs are almost commodity. You have lots of empty DIAs all over. If one box fails, we move over to another box. You need an LDAP configuration server and some kerberos servers, and from there, you can bring up or take down servers spread around the country. We just need more services.

Do you put more than the active mobile objects in the space? The reports that these objects generate? Is there some integration with other software so that people don’t have to learn new software? There’s architecture and reality: in reality, yes, we’ve got adapters to interface with heritage services and how to map between the two.

Evolvable JavaSpaces: you’d have a version 1 client tied to the version 1 entry, and a version 2 client tied to a version 2 entry. Can a version 2 client read version 1 and 2 entries? Yes, it can. When you’re just Jini, connectors between service avoid this problem, sorta. But once you start using JavaSpaces, you have same names, same entry factors, and bizarre error messages if you have version conflicts. It’s very very important to solve this versioning problem if you want to survive. Worry about rogue code into your system introducine toxic objects, too.

We have secure and non-secure deployments. In non-secure (development) environments, you can telnet into a server, use LDAP for authentication. It connects to the LDAP server, downloads its location, it telnets in and brings up its own VM. Then it launches the child phoenix and lookup sevice withhin that environment. Then the child goes through a similar sequence. In the secure invironment, the client talks to a kerberos service. I like what I see with Davis, and we’ll start using that soon.

What are the challenges? What are not very good and should be changed? It seems everything is wonderful and nice, but I imagine that when you go to deploy globally, that there will be problems. I love this question. Last year the main problem was security. We didn’t have security. I now believe that security is in hand. Now, nobody can say how to build a predictable network of this type. I don’t really now how the services will behave in a large deployment. It seems like we don’t need a lot of bandwidth, but who knows. Delays and leasing don’t get along well. We need a tool to put all this model together and show a refernece model of the network, do some predictions, then use that info to ask for money to build it in reality. “Frankly, I don’t know how many X we need, nobody’s ever built one.” Kinda hard to say that in a budget meeting.

Channels through the JavaSpace: FIFO? Yes, Yes, the channels are FIFO. One of the sevice in the DIA is a management service that looks at the load of the other services, and if it sees overwhelmed service, it launches another copy of that service. We have a lot of adapter services that know how to talk to various devices. We have services for specific kinda of data for specific kinds of tasks. We can deploy new kinds of servicecs without afecting other services. There are so many ways to approach thiis problem, so we’ve tried things, backed off, tried a different thing. Just write a service annd see it it works.

What were the fundamentlal problems with J2EE or CORBA systems? The evolution of some of the Jini features are now creeping into other applications. Mature platforms like that will pick up some of the Jini features. For CORBA the problem was the interoperability of CORBA and other products just wansn’t ever solve. 50% of all CORBA projects I ever launched failed. CORBA didn’t have the failure/recovery smarts we need. Difeferent ORB vendors didn’t tlak to each other, even different version fromt he same vendor. Intolerable in a distributed netowkr. Just not flexible and survivable like Jini.

Monitoring and Managing Java

EMMA is a new experimental project at Sun that creates Yet Another Management Standard and Isolation/Unification Layer. You plug modules into EMMA, for your device, for SNMP, whatever. Then management software talks to EMMA, through EMMA APIs, through SNMP, whatever, to get and set management schtuff.

Acronym means blah blah Extensible/Enterprise Management, changes depending on who you ask. Emma is an experiment at Sun to create Yet Another Management Standard. You write an “MBean”, pass it to an Emma compiler, and Emma spits out various management files for different management systems. Knows how to deal with SNMP, JMX, other management systems.

No notes for this session, because my hands were busy shoveling lunch down my throat.

[zz Given its new/experimental nature and complete and total lack of current support (and sparse session attendance), I’m happy to ignore Emma until it gains more traction.]

Java and the Sun Open Net Environment (Sun ONE): a Technical Overview

Sun ONE is Web Services and Web Services on Demand. It is a way to define modular software and then assemble those modules with Forte into web applications.

Focusing on web services today and services on demand. Service is a modular, well-devined enncapsulated function. Used for loosely coupled integration between applications or systems. Based on XML, transpored in Synchronouse RPC/SOAP, Asyn messaging (SOAP and ebXML. Sometimes advertised through UDDI, ebXML RegRep, WSDL and other standards.

You can assemble small services together into a higher level service. You can assemble applications out of services. They're very flexible and easy to use and transform into new forms.

The Impact oN softwrae: Application Dis-integration. A monolithic applications, system software, computer, all break into application services, system services, and the network.

ebXML is an emerging B2B standard. It is secure XML messaging, registry. SOAP ofver HTTP isn’t secure nor reliable. ebXML rides on top of SOAP, reliable, secure.

Services on Demand is an umbrella concept for delivering services any time, anywhere, on any kind of client devices. Includes interneet web application delibery today, emergin infrastructure for web services, specifications for access by current and guture deplyment like J2EE, J2ME, Jini, JXTA, grdi computienr, other stuff.

Sun ONE web architecture includes interoperability, legacy systems, Microsoft interoperability and so on. Include identity/context serivcces like libery alliance. Other middleware interfaces like CORBA. Decomposing other traditionall things like meail into services.

Two audiences for Sun ONE. Developers of services on demand (writers of web applications integrators of services), and software vendors integrating their produce into the Sun ONE stack. Applications in the service container, components filling gags, database, tools, contents managemenet systems, and so on. Serivers ,other major platform components replacing Sun products.

A Framework for Assembling Client Applications from Modular Components

Do not write monolithic applications. Break the project into components. Have each engineer work on one component, and then glue the components together into an application. At the end you still have an application for your customer, but now as a side effect you have a collection of potentially reusable components for future projects. Write your library classes to make reuse and extension easier, not harder. Move decisions about behhavior out to configuration files, rather than compile-time controls like constants and overrides. Layer your library and dependencies to keep your class dependencies managable.

A good module has a decen empty default implementation for most methods: you don’t want to have to override a lot of abstract methods. Give a lot of hooks throughout the whole lifecycle of the applications so that subclasses can override appropriate pieces.

You don’t want to force developers to manually write/copy/paste stuff into resource files.

A problem in frameworks: looking only at the API, you don’t know when each methd gets called. You don’t know if you do or do not want to do heavy lifting in any one function, unless you can run and probe the class within the framework. You have to fall back to documentation.

We put a lot of options in the confugruation/properties file so that you don’t have to do that work in code.

Bad: monolithic application development. Have a team of developers develop The App.

Good: break The App into a bunch of components, each engineer works on a component, tests it, and then we bring it all together as an assembly of components.

You still end up with an application, but now you also have a library of reusable components, too.

Lessons Learned:

Need better intermodule synchronization, so that changes in module A get reflected in module B.

Layers in library

Dependencies OK on lower layers, never on upper layers (standard library snotball-reduction stuff)

Documentation

If you wrote the framework, you know how to use it, but nobody else does. Broke out API documentation, guides to features, feature requests, build instructinos, histories of all the classes in any one layer. There are commercial products that do this, check them out on the show floor. If you don’t do this, you have a lot ofpeople writing code that nobody ever uses, becuse nobody knows it’s out there.

They have one engineer act as a librarian, keeping a website of library class documentation up to date.

Summary

Make behavior data-drive to minimize the need for subclassing and override just to change simple hehavior. Minimize abstract methods, provide reasonable default behavior.

Do not build monolithic apps. Do not build hundreds of tiny applications (you lose your users). Combine related modules into applications.

JDOM Makes XML Easy

JDOM is an open source project and now a JSR to make working with XML easy in Java. Existing SAX/DOM implementation is too hard to use, fails to take advantage of Java and its libraries. Use JDOM to read input from files, streams, SAX parsers, process the JDOM if you wish, and then output to files, streams, SAX parsers. TRaX provides XSLT support for JDOM.

It’s an open source library for Java and XML data manipulations. This session provides a technical introduction to JDOM.

Goals

JDOM is a programming model to represent XML data. It’s similar to the DOM but not build on DOM or modeled after DOM. It’s an open source project with Apache licesne and 2000+ list subscribers. It’s JSR-102. You can use this in commercial code and not worry about GPL viral stuff.

JDOM was created to be straighforward for pgrammers using Java technologyl. We did this for *you* the java programmer. We didn’t want tothink so hard about XML manipulation. We wanted to hid the complexitys of XML wherever possible. It integrates well with SAX and DOM. We use Java features, DOM cannot (it’s language neutral).

SAX doesn’t let you random access, change the docuemnt. SAX is tough to use, requires state machine, stack. DOM isn’t quite right, either. Defined in IDL and lowest common denominator across languages, so it misses out on some really useful Java features.

What Can JDOM Do?

One of our metrics: can you figure this thing out without reading the JavaDocs? Most of the org.jdom package is just XML things you already know: Attribute, CDATA, DocType, Element, Text, and so on.

JTreeOutputter is a class that constructs a JTree out of a amn XML document. Cool.

How does it relate to JAXP? JDOM supports the JAXP 1.1 API, builders can use any psrser, but JAXP parser is the default. If JAXP isn’t in your path, it uses xerxes.

TRaX/XSLT transformations in the JAXP API., use JDOMSource and JDOMResult classes.

How?

Documents are org.jdom.Document. Construct ond from scratch: new Document( new Element( “root” ) );

// basic XMLOutputter outp = new XMLOutputter() outp.output( doc, fileOutputStream ); // or strip whitespace outp.setTextTrim( true ); outp.output( doc, socket.getOutputStream() ); // or pretty outp.setIndent( “ “ ); outp.setNewlinse(true); outp.output( doc, System.out );

The Element Class

Element root = doc.getRootElement(); // get a list of allchild elements List allcChildren = root.getChildren(); // get only elements with a given name List namedChildren = root.getChildren( "myName" ); // get the first element (with name "myName" ) Element child = root.getChild( "myName"; );

The list is LIVE: you can change the list, and it directly changes the document!

namedChildren.remove( 3 ); // removes the 4th child from the DOM allChidren.add( new Element( “jane” ); root.addContent( new Element( “jane” ); // same effect as previous like

While constructing the DOM, JDOM does check correctness: you can’t add a space node where illegal, for instance. Add/Remove methods chak structure, so you don’t have loops, or multiple root elements, and so on.

Attrbiutes

String width = table.getAttributeValue( “width” ); // returns a String int border = table.getAttribute( “width) ).getIntValue(); // returns an Attribute object, that has a a getIntValue() method String desc = element.getText();

String desc = element.getTextTrim();, since you don’t want all that icky whitespace.

getTextNormalized() also normalizes spaces within the text down to a single space, scrubs away all those tabs and whitespace formatting.

element.setText( “alksdfjasdfa” ); element.setText( “<xml> content” ); // becomes <xml> content on output

Processing Instructions

<?br?> <?cocoon-process type=”xslt”?> String target = pi.getTarget(); String data = pi.getData(); String type = pi.getValue( “type” );

Namespaces

JDOM supports namespaces natively. It was created after namespaces were finalized, so namespace support is native, not bolted onto the side like other APIs.

Current Status

JDOM is in Beta 8, released this morning. SPath on JDOM is quite far alon, http://www.jaxen.org. XSLT support is provided by TRaX and incorporated in org.jdom.transform. We would like in-memory validation. We don’t yet have this. Any volunteers want to help out?

Threads, Concurrency, and Synchronization

Java gives you tools for synchronizing data between threads. Use them, but first know what you are doing. Use higher level thinking and code rather than just blindly tossing around synchronized blocks. Don't be clever. You are probably being wrong. The compiler and processor can reorder code, optimize away your cleverness. Avoid holding multiple locks: you are just asking for deadlock trouble. Volatile can often avoid the need for synchronization. Volatile, synchronize are a lot cheaper than you think: do not be afraid to use them. Thread interruption is a cooperative process: async thread stopping is going to leak resources and leave things in funky states (and is deprecated for this reason). Threads behave differently on different platforms. Test your concurrent program on multiple platforms, and at least one multiprocessor platform. Write safe immutable classes: it is possible to write mutating immutable classes.

The current thread specification is broken, JSR 133 is working on fixing it. Revision is fairly close to existing. The new VMS like HotSpot are already compliant with fixed.

In many cases, far more effective to use higher level concurrency abstractions, but sometimes you need the low level stuff.

Synchronization

Synchronization is subltle and confusing. This isn’t unique to java, just life with threads.

Each java object has an associated lock. You do a synchronized(obj) { ... } to acquire the lock for the duration of the the code block. Locks are autmatically released when the scope ends, and are recursive. If you lock on A, and the same thread locks on A again, that’s OK, And the lock doesn’t unlock until all locks are unlocked on that thread.

Everything before the unlock in thread A is visible to all other threads before the other threeads’ lock, Even stuff that got written before thread A’s lock. This means you don’t have to hold locks while performing all of your writes. You can prepare the object first, then hold lock whild adding to a shared buffer. The association between the object locked and the data changed is entirely up to the programmer. A lock on an object does not prevent other threads from modifying that object. Lock is just thread control, not access control.

Confusing

The compuiler and the processor can optimize away and remove synchronization on thread local objects. synchronized ( new Object() ) { ... } is a no-op. Compiler can move/reads/writes into synchronized blocks, but generally not out of a synchronized block. Other surprising transformationsna dn optimizations.

Use lock ordering if acquiring multiple locks. You must avoid deadlock. Think twice and thrisce before holding locks on multiple objects at the same time. Deadlock potential! wait only gives up lock on one object at a time. Can block other threads for a LONG time. Try to aboid holding locks whild sleeping of performing blockingI/P, or calling code you don’t control.

Be real careful about what you do while holding a lock. You’re just asking for trouble.

Volatile fields

If you are going to access a shared field without using synchronization, it needs to be volatile. Semantics for volatile have been strengthened in JSR-133. Sun HotSpot 1.4 is already compliant. If you don’t try to be too clever, declaring it volatile usually just works.

Volatile works for one-writer/many-reader models: volatile boolean done = false; This works best for monotonically increasing values: flip back and forth and you could miss a change. Volatile works well for distributing a recent-but-not-necessarily-current value of something. Code that needs the most recent value won’t work. Needs synchronization. Incrementing a volatile field does not work. If multiple threads perform updates, two threads read, add 1, write and each thread writes N + 1, the resulting value is N + 1, not N + 2.

It’s impossible to keep two volatile fields in sync. Use synchronization for that.

Interrupting Threads

There is no way to force another thread to stop. You must ask nicely with Thread.interrupt(). Other thread must be prepared to be interrupted: be interrupt-friendly. You don’t have to write every little method to expect cancellation, just in longer-running functions and loops. All that thread.interrupt() does is set an “interrupted” flag in the thread. Then your thread must check that flag.

static boolean Thread.interrupted() returns true if the current thread is interrupted. As a subtle side effect, it clears the interrupted flag. Only the current thread can clear its own flag.

boolean Thread.isInterrupted() does not clear the flag, lets you query the status of any arbitrary thread, and is generally a useless function you will never call.

Certain methods can throw InterruptedException: wait, join, sleep. Some I/O methods could throw, too.

The new I/O libraries let you not interrupt, but close a stream/socket, so the interrruption puts you into a known state.

Rethrow the interruption. Make the interruption part of the method’s throws clause. Or re-interrupt yourself. you really don’t wnat to hide from higher-up callers that you’ve been interrupted: they really may need to know.

Thread.currentThread().interrupt() will re-interrupt you so higher-up code can do proper cancellation.

Using wait/notify/notifyAll

Allows threads to wait for a condition to hold, until informed by another thread that it may hold. notify informs a single waiting thread, notifyAll informs all waiting threads.

Importrant: must hold the object’s lock before using. Protects access to the condition being checked. Lock is released while waiting. The state you’re waiting for must be protected by synchronization, because the piece of state that you’re checking (for wait) or changing (notify/notifyAll) must be locked so it doesn’t change out from under you. wait() releases the lock on the wait object (otherwise the object could never acquire the lock and notify, ne?)

class OnePlaceBuffer { private Object o; public synchonized Object take() throws InterruptedException { while ( o == NULL ) wailt(); Object result = o; o = NULL; notifyAll(); return result; } public synchronized void put( Object v ) throws InterruptedException { while ( o != null ) wait(); o = v; notifyAll(); }

You must always wait in a loop! Cant’ assume that just because wait returned that your condition changed to what you expect. At worst, you’ll have tiny overhead. No loop, and at worst you’re continuing in an invalid state.

When notify, when notifyAll?

notify is an optimization: use only when only one thread could possibly benefit from the state change. “I put only one thing in the buffer, only one thread can use the buffer, so notify only one thread.” The problem is that if you have multiple threads waiting for conflicting states, notify might always notify the wrong thread, and you never wake up the thread you want.

Cost of Synchronization

Synch isn’t free, but its pretty cheap on modern VMs. Cost really a factor only when wrapped around small methods. In most problems, synch isn’t the bottleneck. The cost of synch is much less than the cost of stupidity. Few suffer from the cost of synching shared accesses. But some classes synch all access, even when not shared by threads.

Classes that synch all access: 1.0 collection classes (the old collections: Vector, HashTable) They’re oversynched and never do your program any good. Use ArrayList, HashMap instead. If you think you need synch, synch your blocks intelligently. Any concurrent program should use java.nio, java.io is icky. StringBuffer has some funky synching problems, it can hurt.

Avoid the need for synch

Constrain objects to a single thread. Use ThreadLocals. No changing state means no interference: stateless objects, immutable objects. Declare fields final. Containment restricts access. If the container’s synched, you don’t need sync on the contained objects.

ThreadLocal

Allows you to create a variable with a separate value for each thread. Never needs synch. Can be used to cache values from shared structures. ThreadLocal’s are much faster now, JDK 1.4 makes it about equal to a method call.

Don’t be overly Clever

People sometimes try to carefully reason about cause and effect and ordering to find a way to communicate between threads without using sync or volatile. Almost all such attempts are wrong: e.g. double-checked locking. Compilers and processor do reordering and transformations that thwart such reasoning.

No Promises

Priorities may mean nothing. Preemptive multithreading might not exists, yield/sleep is only way to switch. Yield/sleep may mean nothing. In decent VMs, you will get some reasonable behavior, but don’t depend on the details!

Test your concurrent programs on at LEAST one multiprocessor platform, and different platforms if possible.

Transactions

Be careful about relying on the build-in synch of library classes. Often you need to perform a series of actions atomically:

The problem is that the individual steps are synched, but you need high-level synch to do this right. Another thread an come in between steps 1 and 2 and insert the element.

Finalizers

Just don’t use them. They’re painful for your VM, painful for you, hard to think of a case where multiple threads access the object where the finalizer may run wile components of the object are still in use.

Safe Immutable Objects

Immutable objects can be subject to attack! One thread creates aobject, passes reference to object to another thread via a race, second thread might not see all writes done by constructor. Don’t use the immutable object before its final fields are set. Don’t register yourself in your constructor with any collections (like a global registry or hash) until you’re fully constructed.

Q&A

Security manager won’t let you interrupt an applet. Do you have to sign the applet? Unfortunately, that’s life. Security manager can get in your way.

Multithreaded app that needs to be fast, one thread does all the work and wants to be fast, the other thread just wants to get statistics out. I don’t want to put thread contention in just for stats. Can I avoid synch here? You should use volatile. If you don’t, the compiler is free to hoist that poll out of the loop, you’ll never see the data. On most processor, volatile is just a memory access: volatile is often a hint to the compiler to “don’t hoist this out of a loop”.

Why call notifyAll right before exiting a synch block? Why? notifyAll is more than releasing the lock. You can actually do the notifyAll as soon as you know you’re OK, notbody will be able to go until you release your lock.

Networking with J2SE, Present and Future

J2SE 1.4 now supports IPv6. New Inet4Address, Inet6Address subclasses of InetAddress implement their respective addresses. All the old APIs and code that take InetAddress objects work with IPv4 and IPv6 addresses. Only code that assumes 32-bit IPv4 address formats will need to change. New unconnected socket APIs let you configure a socket before binding/connecting. New URI class supports any scheme, not just the connection schemes that URL supports. New class NetworkInterface represents a network adapter, lets you iterate over adapters, support multihoming. Improved SOCKS support makes working with proxies better.

This talk is all about the classing networking in java.net.*, not the new JSR 51 I/O stuff. But lots of changes in 1.4 that you still want to know about.

J2SE 1.4:

IPv6

Benefits: expenced routing and address capabilities, auto-configured addresses, authentication and privacy cpaabilities. 128 bit addresses, not 32-bits.

JDK 1.4 supports RFCs 2373, 2553, 2732, 2893. Only on dual-stack systems, (solaris, linux with IPv6 support compiled into kernel). WIndows uspport later when dual stack available. It’s all automatic: IPv6 is detected at astartup time, code change required only for applications that assume IPv4 address format.

API support

New classes Inet4Address and Inet6Address, subclasses of InetAddress. InetAddress.getByname() and other static methods now return subclass of InetAddress, not just an InetAddress. All the old APIs that take an InetAddress with work with inet4Address and Inet6Address. It just works. Support adde throughout the rest of networking API, including the new I/O DatagramChannel. java.net.URL/URLConnection all work with IPv6. Most APIs and code are IPv6 without changes.

Unconnected Sockets API

Bad name, covers more than unconnected sockets. More flexible extended socket API. Frequently request features. Sets sockets options prior to binding, or connecting. Specify timeout when establishing a connection. New classes, constructors, methods.

New class: SocketAddress

An abstract class that represents a socket endpoint. Used for bind/connect operations.

New class: InetSocketAddress

Extends SocketAddress, pair< HostAddress, Port Number>. Allows for unresovled addresses.

New class: java.net.URI

What’s the diference between URL and URI? URI is scheme-independent, URL is dependent on the underlying protocols for the schemes. This is due to URL.openConnection(): you can only create URLs for protocol that can open a connection. URLs have to be openable, and thus complete. URIs don’t, and thus can be partial, such as when relative to a separate URI. You still need to use the URL class when connecting to resources.

New Class NetworkInterface

Represents an actual network adapter such as en0. Allows enumeration of network interfaces and addresses bound to each interface.

// examples NetworkInterface.getNetworkInterfaces(); NetworkINterface.getByName( “en1” ); MulticastSocket.joinGroup( SocketAddress, NetworkInterface );

Enhancements to Old Class: URLConnection

URLConnection was th esourcce of man issues prior to 1.4. In 1.4 the protocol handlers are significantly improved. FTP now supports getOutputStream() so you can “PUT” to server. Passive mode FTP now supported (PASV). Improved proxy settings. HTTP error handling much improved. getErrorStream() returns stream with entity body when http error returned. Digest authentication (RFC 2617): more secure than basic digest authentication. Allows the client to authenticate itself to the server, and server to authenticate to the client. Streaming of chunked encoded responses. Improved keep-alive. Oh yeah, lots of bug fixes, too.

Improved SOCKS support

For TCP client and server only. SOCKS v4 and v5 (RFC 1928) support. Auto negotiated with proxy server. Unresolved addresses supported: you could have unresolved addresses if you’re behind a firewall, no DNS access. Transmits the hostname to the proxy, the proxy resolves. System properties socksProxyHost and socksProxyPort control all this: if both set, all sockets are created using SOCKS.

Improved Name Service Support

By default InetAddress uses the system configured default naem service. But now with 1.4 you can bypass this mechanism by setting a system property to use a java-based name service provider. We provide one based on DNS and JNDI. When would you use this feature? If your machine uses multiple name resolvers and you want to be specific. You also give InetAddress control over result caching.

JSSE Java Secure Socket Extension

Adds API support for secure Socket communicatinos using protocols such as SSL/TLS.

The Future

URLConnection/HTTPConnection have limitations. We see a need to improve HTTP support in the platform. We need a new HTTP client API. We could use that new non-blocking I/O in JDK 1.4. Need full compliance with RFC 2616 HTTP support, we don’t fully support the RFC. We can improve HTTP extensions like WebDAV. Raw socket support, many utilities folks request this [zz that would be cool].

Friday

Unit Testing Made Easy: Design Principles to Facilitate Testing

Unit testing helps you have confidence in your code. The very act of writing a unit test will affect the code it tests: it will improve your design and force you to decouple your classes from each other. If your class is difficult to test, it is probably written poorly. There are tools to help you test some difficult code such as threaded code. Mask and Mock Objects are a common pattern of surrounding a tested class with fake/stub objects that are actually part of the test suite.

Why unit test

confidence in your code - if you’ve got a test suite you can run on a class, you say “it’s good”

refactorability - refactor your class and run the test and know you didn’t break anything.

interfaces designed from perspective of generic client - gets you to look at the API for your class, not just the one use you were thinking of when you started the class.

unit test: you wrote your code correctly. Functional test: you wrote the correct code.

http://www.delphis.com/java - a JUnit cheat sheet that gives you 90% of what you need to know about JUnit, all on one sheet of paper

He creates parallel source and test FILE hierarchies, but not PACKAGE directories. Unless you expect yoru stuff to get subclassed, you don’t need package/protected access to your class usually. When you’re subclassable, you need tot est what subclasses can get to.

Hallmarks of good unit test

You want your developers to run these several times an hour. You want to keep passing tests at 100% so that the system never gets wildly off track.

How do I test this Stuff?

Overachieving constructor: a constructor does too much, too coupled to too many other stclasses. Singleton creation, helper classes, thread spawning, it’s all a pain. Break the constructor into constructor + “begin” method, make your constructor take in arguments rather than construct them itself.

Jack-of-all-trades class. A class that does moe than one thing. A ProtocolSession class that actas as a state machine and a message sender/receiver. Break into two Interfaces that it implements, and then plug different impelmentations of those interfaces to test sttate machine and message send/receive separately. Use a mask inner class tin the test that implements the interface.

[zz His mask code (stub interface implementations) sets data members directly in the Test class, makes the actual test case code a bit simpler than what I’ve been doing. Nice. Better than my usual mask code.]

Insufficient abstraction of external resources: business logic tied too closely to external resource like JDBC. it’s impossible to unit test without including that external resource. You need to separate the resource: location, acquisition, use. You can separate out location with aproperty file or something. Acquisition can use a pool or something. But the use of it? Write your code to some high level interface, and then let your unit test implement that interfcace.

Multi-threaded code. Ugh. COmplicated notify/wait logic. Refactor to encapsulate notify/wait. ThreadSpawnableTest Decorator. Use libraries like Doug Lea’s concurrency utilities:

GUIs: notoriously hard to test. Because responsibility is wrong. Need small enough units to test, so you must be MVC. Once MVC, logic is moved away. To make unit testing easier, we make the design better.

GUIs: the model’s already tested. Provide a modk model. Test connections betweek GUI and model. Does menu initialte correct model call? Does Model update display in JList? Layout important? Test relative positions and existence of GUI components. Test sequences of actions, do parts of prior actions leave junk in the UI fields and such?

Singletons

Beware of singltons: often abused as global variables.. You get too tied to getInstance(), that one global. Hard to get around it. One way around is a setter in the singletone, but if you’ve got a setter, it’s not really a singleton, it’s just a global variable. YOu’re just increasing coupling. You can have limited places where the singleton’s getINstance is called, and just pass pointer around in most cases. Then you’re better decoupled.

When do you REALLY need a singleton? Not often. Maybe a logger, or a physical object that really only has one instance.

Mock Objects

Souped up masks, they’re separate helper classes with actual logic. You can set them up to throw exceptions on certain method calls: important to test exception handling! Set number of times a method should be called. Provide canned data. Varied data can give you better coverage in your tests. They promote loose coupling.

Inheritance vs. Composition

Inheritance invites Fragile Subclass problem. Modifications to superclass breaks subclasses, because subclasses usually depend on superclass implementation. Inheritance has its uses, but don’t overuse it.

Composition promotes Builder classes: classes that glue together sub-units at component or application level. Composition avoids class explosion: lots of little subclasses each with slightly different behavior.

Smaller classes are eaiser: to write, to understand, to test, to optimize (if profiling says you need to).

Exposed vs. Hidden Composition

If you have a class that is completely enclosed by another class: you never touch it directly exception through another class...you don’t need to test it: it’s tested by the test for the container class.

Using unit tests is nothing new: writing test promotes all the good stuff we know about already:

A college professor is now asking students for unit tests in additon to the code that completes an assignment. The result the students are learning better design.

Extreme Programming

You can’t talk about unit testing without at least mentioning extreme programming.

Doing it all the time takes discipline. It makes unit testing a design tool: unit testing creates good design, XP is not hacking.

Tips

Know your tool. JUnit is a framework, you can get inside it, extend it. TestDecorators. if you find yourself writing the same code over and over, setUp/tearDown! Subclass TestCase to make tests easier.

Test For These!

Keep your tests up to date

As soon as you stop running the tests, the design starts to drift, bugs creep in.

When you find a bug, DON’T FIX IT. Write a test to expose the bug, then change code to pass the test. Now you’ve got a regression test and you’ll never see that bug again.

Incorrect documentation is worse than no documentation. unit tests act as documentation. They won’t be incorrect unless you stop running them.

Q&A

Compare unit testing with design by contract? I haven’t really gotten into ddesign by contract, you’ll get a lot of the same benefits. But I think unit testing is more approachable.

Is Junit practical for Swing component testing? JFCUnit add helper classes that make it eassier

JUnit for functional testing? There are some people doing that, I personally do not. It’s such a generic framework, it’s possible.

Code Coverage tools? They’re great. Especially if you write your tests after the code. If you write tests before, you’re already covered, though. When retrofitting testing onto existing code bases, then coverage tools can help. There are Mock object generators, too, which can help you fake up scaffolding.

How develop tests for old crusty code: curse like a sailor. No simple answers. Test new functionality, bearable amount of pain. When you find a bug, right a test to find it. You can’t do a big bang all at once: developers will quit.

Any recommendations for JSP testing? junit.org has results of lots of people doing this. HTTPUnit, J2EEUnit.

Using ClassLoaders to Support Better Design, Security, and Hot Swapping of Components

You can use the class loader classes built into Java (especially URLClassLoader), or write your own. Use these class loaders to control where your code comes from, what it is permitted to do. You can swap in new loader objects while running to introduce new class implementations to your running application: dynamically swap in new class implementations without restarting your application.

Class loaders can be a trememndous way to structure programs to be easy to debug, easy to maintain. This is a lession we learned accidentally, and ever since we found this out, we’ve created our applications using class loader trees. Integrating classloaders in program design can double the efficiency, security, and maintainability of your applications.

Class Loaders

They load classes (duh!). They can be structured as trees: each class loader delegates requests to its parent before trying to satistfy it. They read classes from the file system, network, or generate byte code dynammically. In practice, they used by security managers ti indicate security domains.

JDK 1.s introduces new features that make it simple to program class loaders. JDK 1.4 add new features to support assertions.

ClassLoader clA = new MyClassLoader(); Class classB = clA.loadClass( “B” ); MyInterface b = (MyInterface) classB.newInstance(); b.doSomething();

Create Your Own Class Loader

class MyClassLoader extends ClassLoader { public Class findClass( String name ) { byte[] b = myLoadClassData( name ); return defineClass( name, b, 0, b.length ); } private byte[] myLoadClassData( String name ) { ... } }

By using different loaders for different parts of your application, you can place differing seciruty restrictions, and thus good design, on your classes. This part of code can access the database, but not the screen, this part of code can do the UI, but cannot touch the database. This part of tcode cannot spawn threads. And so on. He uses an XML file that defines each loader, what classes it loads, where it gets them from, and what is allowed for that loader’s classes. This is all statically defined in an XML file. In JDK 1.4, you can change these permissions dynamically.

In the loader hierarchy, put interfaces and factories in parent loaders, and implementations in child loaders

Hot Swapping Classes

If your program takes 5 minutes to start up, you don’t want to restart it every time through your compile/edit/debug cycle. So create a class loader (even a simple URLClassLoader) that points to the new file/new class. Then remove the class loader that has the old instance of the class. You need to remove all references to the old class, such as closing windows or nulling things out.

JDK 1.4

void setDefaultAssertionStatus(), setPackageAssertionStatus, setClassAssertioStatus, clearAAssertionStatus(). [zz this relates to loaders how?]

Java Web Start: you cannot create loaders unless you are in privileged mode. You cannot do loading in a sandbax. Java Web Start’s loader doesn’t do URL loading.

JSR-121 Application Isolation

Understanding the Java Logging API

JDK 1.4 offers a new logging API. It looks a lot like log4j because log4j did a lot of good research and work on java logging. Everything log4j does is still possible with the new logging API. The new logging API is an open framework. You can extend it and swap in new components to meet your needs. Logger objects do all the work, are the only class you are likely to interact with, anologous to log4j Category objects.

[zz This is the third straight session that was packed with people and spilled over to an overflow room. Either all the other sessions at this time are lame, or everybody wants to be where I am. I guess I’m more popular than I thought. :-) ]

System.out.println() is wrong. Use a logging API. The API is even easier to use than println().

Excuses: no standard, weak or underdeveloped APIs, libraries only route to stdout, poorly documented, too involved. What’s the point? I had to develop 4 or 5 classes just towrite my first line of logging output.

Logging is a lot mroe complex than most people give it credit for. The JDK 1.4 logging API is a full-featured framework, extensible, you can plug in your own implementations for just about anything in this framework. You can still be stnadard while doing it, too. If you have a 1.4 JRE, you jave the logging API. It scales from small to enterprise uses. J2ME, uh, no. Sorry.

The Players

Each logger has a hierarchical namespace, just like the package namespace hierarchy. You don’t have to do it this way, but you’l want to.

public class SimpleLoggging { pricae Logger logger = Logger.getLogger( "com.sun.javaone.logging" ); public static void main( String[] args ) { logger.fine( “hi” ); ... or try { throw new Exception() } catch( Exception err ) { logging.log( Leve.WARNING, “Exception in main”, err ); } }

There are multiple methods for entering message. At runtime, they tetermin calling class and method. HotSpot does a lot of optmizations, entire stack frames can disappear, couse they’re not needed. If you see a message from a method without logging, maybe it calls a method with logging, and hotspot merged you.

Handler

The most common thing you write yourself. How you add custom message delivery to logging. Handlers are controlled by the logger objects. THey are not some global thing that all logers pick up. Loggers are assigned a handler at start. Handlers do not act independently. Handlers do not ever deal with content. That’s Formatter’s job. You don’t know where tha message will go after you: it might go to another formatter, or it might be forematted before you get it. Don’t muck with the data.

Handlers have a level (usually supplied by the controlling Logger). Message with a Level of a lower value than thhe associated Level are ignored. Handler.publish() is where teh message goes out. You can do level comparison here, or in the Handler’s Filter, but he’s never seen it done that way.

J2SE 1.4 has 5 Handlers ready to use. It should be rare that you need something else

Formatters

Takes a LogRecord and creates a String. J2SE 1.4 includes 2 default Formatters. You can to internatilalized log message: this could be very powerful.

LogManager

LogManager handles maintenance of the hierarchical namespace, methods for retrieving and setting confifguation information, manage the global Handler objects. It’s implementd as a singleton and is safe for EJB even though it’s a singleton. You can replace it with a custom implementation if necessary.

Configuuration

Controlled by logging.properties in the JAVA_HOME dirrectory or passed in on the command line -Djava.util.logging.config.file=myfile.config. Or you could set java.util.logging.config.class system property to a class that do the configuration.

Security

The logging API uses a special permission class called LoggingPermission. All trusted code is given a LoggingPermission. Untrusted applets are allowed to create Logger objects, bu cannot change configuration settings. Untrusted applets can create anonymous loggers. It doesn’t have a bansespace associated with anonymous loggers. They have no access checking, so they can do anything they want. They can load your file system with log messages. They can change their configuration settings. Not recommended.

Other Gotchas

LogRecords do not have unique identifiers. There is little synchronization in the API, occcasionally resulting in messages arriving out of order. There is no prevention of recursive logging loops.

Log4j

I love log4j. But the JDK 1.4 is standard, don’t have to add a jar. The Sun API offers more functionality. You can pass arbitrary objects with message. I can’t imagin anything in log4j that you can’t do in the Sun API.

Q&A

Log viewers? None that I know of, but with the XML formatter, it should be trivial.

A Programmer Survivor’s Guide for Java Using Mature Programming Practices, Patterns, and Principles

Errors are fact of a program’s life. Write your program expecting errors. Write high-level error handlers to catch and deal with errors. Write low level error handlers only in extremely rare cases, and only if you can articulate a very good reason on. Low level error handlers often mask useful low level information, incorrectly guess high-level intentions, and cause copy-and-paste code duplication.

Civil Engineers define safety factors: the elevator wire had to handle so must capacity, and a secondary wire carrries more. We need to start applying failsafe mechanisms to the code we write. Protected memory in real operating systems is one form of this.

Be prepared for anything that can go wrong. Even methods that don’t declare exceptions can throw them.

Exceptions

All errors become exceptions, rather than crashing the system. Try/finally constructs rocks. There is no excuse for java programe to be anything less than rock soleing

Note, even though we say “excception”, we mean “Throwable”: deal with all thrown errors.

If you catch errors too low, you still have to catch at high level. So low level handlers are wasted effort. Now your recovery code is scattered and duplicated. And low level handling might have to second-guess what high-level calling code really wants.

Higher-Level Recoery Strategies

The actual class of the error in the log message is irrelevant: but the message mattes. Don’t mask lower level errors with vague higher level handlers.

Add more specific handling, but only when absolutely needed! There should be a difinite reason why you’re adding this handler. If you don’t distinguish between error types, you won’t be surprised when something new shows up.

Generate Exceptions

Scope: Local Errors

Some can be foreseen. Could arguably be part of the interface. But it’s really kind of extending the API, through an ordinary return value. Handlers can distinguish by type. Buut if you find yourself writing try blocks with lots of catches, maybe the method’s too big, maybe you need to return a status object instead of a rack of exceptions.

Scope: Distant Errors

Large distance between thrower and catcher. The catcher probably does not understand the error, and the type does not mater. The messaage in the exception is probably the only important thing. Exceptions are excellent for this purpose: you get to send meaning/message across a vast range of code.

Exception Types

Type only matters if it is part of the API. Probably only local handlers are prepared to distinguish between exception types. Reuse existing exception types whenever possible. The mesage is important: get that right, regardless of the type.

Intrinsic Exceptions

You say you don’t generate exceptions? What about OutOfMemoryError, NoClassDefFoundError, ArithmeticEcception, NullPointerException, IndexOutOfBoundsException, and so so many more. Errrors and RuntimeExceptions are also worth handling!

Declare Exceptions

Checked exceptions don’t really work. Failure modes are implementation specific. Exception say more about the implementation than the interface. Immplementation is not stable, it changes over time. You want a stable interface, so you don’t want to have to ripple new exception declarations through code as some underlying code changes.

What to do with undeclared exceptions? Squelch? Wrap? Wrap’s OK. But propagating the original exception is always best. So just say “throws Exception” and then you can let it go. Or you can let it be a RuntimeException and pass through unchecked.

Bulletproof Code

Enforce consistency of state. Strive to dwrite methods that either complete successfuly or change nothing at all. Don’t set some of the firleds but not the rest of them. Don’t get stuck in a bogus state.

When I write a method, I don’t put any error mhandling at all.Then I go through tht ecode and think about what could go wrong, silently,. Then I throw exceptions, catch some exceptions, protect temporaty resource with try/finally. If the computer compials about undeclared exception, “throws Exception” and go on.

Cleanup: a program can always crash despit you best effor, and may leav egarbage files around. It’s good manners for a program top pick up any trash left around from the last run: look for your pollutiona ndn clean it up.

Separate Constructino: do not activate objects in the constructor, use a separate to activate them. Construction is allocation, not activation. [zz this also makes testing easier, too] Handle Weak Input: generate clean output, but be tolerant of lame input. Don’t be fussy if the intent is unambiguous. Log warnings if you detect errors.

Deal with Memory

It’s perfectly reasonable to keep running even after you run out of memory. For example, if a server has many connections, and allocating ana additional connection fails due to resource limits, keep the other connections going! If per-connection memory allocation fails, only that connection dies . App-wide allocations will kill the whole app, though. Give your application a “low memory” mode where it knows it’s low, start closing memory and closing connections to free up memory. Use a rainy day fund to release in this case.

Extreme GUI Testing: A Guide to Using the JFCUnit and HTTPUnit Extensions

JUnit is great for testing most code, but painful for GUI and web UI testing. JFCUnit and HTTPUnit give you utilities to help test your UI. Even though UI testing is really black box testing, not unit testing, it can provide useful bug and regression tests. Once you learn how to drive JFCUnit or HTTPUnit, you can write tests quickly.

There are testing tools out there, but they’re not well suited for unit testing: they’re more for functional testing.

Why do you do unit testing? To give the developer conficens that the class is working properly. It can be used to quickly find the source of bugs. Focuses the developer to think about the design of the class: you end up with better designed classes. Helps refine the requirements of the indidual classes. We don’t even know the right questions to ask when defining a class: testing helps you come up with them.

// automatically run all test methods that start with "test": public static Test suite() { TestSuite suite = new TestSuite( MyTester.class ); return suite; }

JUnit’s Limitations

JUnit does not spearate test data from test code. Cannot bve used to directly test web or applications based on JFC/Swing. Doesw not take the place of functional testing. Does not test performance.

HTTPUnit

Extends JUnit to allow automatic testing of web sites. Emulates a web browser behavior, including form submission, basic HTTP authentication, cookies, and automatic page redirection. Provides methods to examine returned pages as text, an XML DOM, or containers of forms, tables, and linkes. Parsing of HTML is done with Tidy, so it cannot deal well with malformed HTML.

WebConversation, WebRequest, and WebResponse are how you rerieve web pages. You can parse the input stream from the WebResponse, view the page as a DOM object, or a list of WebForms, WebLinks, or WebTables: representations of HTML items in your web page.

JFCUnit

Extends JUnit to allow automatic testing of JFC applications. Provides the JFCTestHelper class that allow test methods to find components of a Swing application. Provides a way to manage the awtThread so that test methods to no interfere with even processing. Presently in Beta release.

The key here is that JFCUnit lets you operate on Swing UI frames even without displaying the UI. That way you don’t have to dispplay UI in your test, and your test doesn’t get slammed by a bunch of “UI not fully inflated” errors.

JFCTestHelper helps you find components within a swing UI, like findButton(), findJLabel(), and so on. It’s not like using the robot to click buttons ans such. findNamedComponent( String, Container, int ). Any component can have a name: this is really useful for testing.

JFCUnit.awtSleep() sleeps your code until the AWT thread does its thing, then wakes up your thread to finish the test (or continue the test).

Best Practices

Write unite tests at the same time as the code is witten. Test early, test often. Make tests easy and fast to run. Make sure the tests capture the requirements: if your requirements change, make sure the tests reflect that. Write new tests as defects are found.

Use unit tests to communicate. If QA finds a bug, have them write a unit test that shows the failure.

JUnit best practices: give each test class a main. Make your cases as independent as possible. Especially for GUI, where you reuse the GUI across tests. Minimize setUp/tearDown to speed up text execution. Use fail() instead of assert() to catch test case errors: the fail method lets you add a useful string. Document your test code: people have to maintain and read test code just like other code.

GUI Testing

GUI testing is a lot different than JUnit testing. GUI testing is black box testing, not really unit testing. So you need a test plan: simulate the user experience, go through the GUI the same way a user would. You need to concentrate on important elements. GUI testing should just test the GUI, not the backend components. If you can’t do 70-80% of your testing through JUnit, you probably don’t have good separation of your GUI and your backend. Can’t test look and feel.

HTTPUnit Strategies

Organize your tests by function. Login, checking airlines, and so on. Keep tests self contained: have a local database and whatever you need to power your web session. Try to create tests thtat don’t rely on format or position. You can’t test JavaScript. You can’t test performance.

JFCUnit Strategies

If possible, use the setName() method to name components int he classes being tested: it’s easier to write JFCUnit tests when you search for components by name. Only test the GUI elements with JFCUnit, use JUnit for everything else. Test at the individual window level, and then again at the main application level.

Q&A

We hate walking the DOM to find stuff in the HTML results. Use XPATH. *Much* easier: one method call with one string, instead of 8 method calls.

Recommendations on testing internationalized apps where the text is localized? I guess you could drive your test with property files.

From a management aspect: it doesn’t appear to be easy or fast to add this testing to a project, and it could be much faster to just test by hand. I’ve used this partially: it takes a few days to get up to speed, and after that it’s not too hard. There was an article in JavaPro, it seems to add about 15% in coding time, and a 50% reduction in bugs. I can write a dialog test in 15-20 minutes.

What happens if a JButton click induces an exception, and your app catches the exception and displays a dialog box: does that hang the test? I guess you’d have to check for that dialog box in your test code. Hmm. You really need a test plan. Good question.

Can I test that my app throws my exception class? Yes, put a try/catch and catch the expected exception.

Running the test in the same JVM as AWT: isn’t this a violation of clean testing? Yeah, I guess so, but I’ve yet to ever see a problem with this.

findButton() verifies that a button’s visible? No, check that manually. Stacked on top of each other? No, you’d have to check their bounds and properties for that.

JFCUnit/GUI testing is really painful compared to back end testing. It takes longer to write the GUI test than to write the GUI! It’s a matter of how you write your tests and your test plan. You have to feel it’s worth it, too. A GUI that you think is gonna be around for years probably benefits from the testing, a temporary dialog you plan on swapping out maybe not.

Can JFCUnit run headless? Yeah, should work. Uh, lemme think about it again. I don’t think so.

The Game of Life Revisited: Managing Parallel Processing Communities with Jini

John Conway’s Game of Life is an appropriate metaphor for how a network of simple autonomous systems can create high level emergent behavior. From a system of simple renderers, XSLT transformers, job controllers, and other services, Procoma created a powerful distributed rendering solution for a banking customer. Emergent behavior produced resiliency, failover, load balancing.

http://artima.com is a Jini site some of you might know about. When Jini first came out, a lot of people figured it was how your lightswitch would talk to your lawnmower and your microwave. That was an underrepresenation of what Jini is about. Jini is not just devices, it’s software talking to software. Jini is still young, we’re all still figuring out what you can do with Jini.

Life

Emergent behavior: the larger-scale behavior is not programmed, it’s a side effect of the smaller rules. This is a metaphor for system rules: if you choose them right, you get emergent behavior. Each cell is autonomous but interactive, with a few simple rules. Complex behavior emerges.

Enterprise Computing Architectures

It seems like each vendor has two, maybe more, architectures for distributed computing. Jini: yet another all singing, all dancing distributed computing/enterprise architecture/application framework? Do we really need another one of these things? Well, look at where these different architectures apply. Jini has some unique qualities.

J2EE assumes that all software will be deployed manually, deploying J2EE software is...uh...hard.

Jini Run-Time Discovery

Assumes transient and spontaneous services, that thing join and leave all the time. It espect failure, failur is not a fatal event. Partial failures are easy to deal with. Request a new provider, and you’ll be told when one comes on line.

Why do we never hear about Jini as an enterprise solution? Mainly I htink because of marketing. Perhaps an attempt to avoid confusion with J2EE. Jini delivers the qualities we desire in enterprise computing.

Case Study: VLS XSLT processing

Problem: create high quality renditions from XML., to different output formats (PDF, PS, TIFF).

We created a prototype, XML in, we apply transformations, generates output. Problem with the prototype: doesn’t scale, distribute, have abstraction layers for external plugins. Hard to maintain. We decided we needed an enterprise application framework.

Two weeks later: every computer in the office became a Jini node, with renderers, transformers, deliverers, and controllers. We predated JavaSpaces, so we had to roll our own. JavaSpaces would have really helped. We had natural load balancing. We could handle very high throughput. We have natural clustering. Rollback on failure is perfect. We don’t have any middleware. We don’t have configuration. We haven’t touched a running system in 3 months: it just works. New renderer? Takes about a day to write, and trivial to add to the system.

Production Experiences

Minimal Maintenance. No service outages. Software update of nodes went unnoticed. “Just add nodes to a network-boot-server and turn them on.” It’s basically an appliance.

Future Work

We’ll switch to GigaSpaces, replace our homegrown cache manager with a real JavaSpaces implementation. We want to embed moe third-party components. We want to allow a process to specify its successor.

A Thought: Self-Organizing Systems

Different usage profiles require adaptive service distribution. We have peak usage times like first thing in the morning when everybody starts work. Can we support peak times in different domains? Give the systems some kind of awareness of its load on its resources?