TestNG

Ovid on 2008-07-16T09:48:29

If you read up on TestNG, you can see an example of their XML output:


    
        
            
                
                
            
            
                
            
        
        
            
                
                    
                        java.lang.AssertionError
                            ... Removed 22 stack frames
                        
                    
                
                
                
                
                
            
        
    

Wow. That's breath-takingly ugly. I compare that to the equivalent in TAP with our optional YAML diagnostic syntax (this isn't how the YAML will really look, but it's just to give you an idea):

1..2
not ok 1 - someDescription2
    ---
    signature: test1()
    name: test1
    start: 2007-05-28T12:14:37Z
    end: 2007-05-28T12:14:37Z
    exception:
        class: java.lang.AssertionError
        stacktrace: |
            java.lang.AssertionError
            ... Removed 22 stack frames
ok 2 - someDescription2
    signature: test2()
    name: test2
    start: 2007-05-28T12:14:37Z
    end: 2007-05-28T12:14:37Z
    xtra:
        is-config: true

The beauty of that is a TAP parser only needs to see this:

1..2
not ok 1 - someDescription2
ok 2 - someDescription2

TestNG has one big advantage, though: it's been written. We're still waiting on Test::Builder 2.0 before we can get the YAML. Fortunately Schwern has a grant for that work and hopefully he's making it happen.


Re: TestNG

jmcnamara on 2008-07-16T17:47:22

Wow. That's breath-takingly ugly.

Yes it is but I doubt that any Java programmer looks at it in that format. They are more likely to look at it (or JUnit results) as a green/red bar in a test runner applet or via some plugin to Eclipse or Netbeans.

In fact you will probably be hard-pressed to find a Java programmer who can tell you anything about the JUnit/TestNG formats apart from the fact that they are in XML.

John.
--