Our whitepapers

Dave Thomas on what is wrong with current programming.

Dave Thomas Interview . Insights on what is wrong with current programming
http://www.drdobbs.com/architecture-and-design/dave-thomas-interview-the-corruption-of/240166688?pgno=3 

A pearl that I really recommend together with The tragedy of Craftsmanship.

and The Coming Software Apocalypse .

This last one is a podcast for the more technical oriented ones: Combating Complexity: Fad, Fashion, and Failure in Software Development

JMH alternatives : REPL time, linux perf

Expanding on previous articles about measuring with JMH

Besides using the calls to System nanoseconds and millis …
There exist some tools for a quick feedback about the performance of your java JVM code.

ON ALL JVM
You can use the time command in the REPL
class Nothing {
public static void main(String[] args) {
}
}

Compile with:
> javac Nothing.java

and run:

> time java Nothing

real 0m0.104s
user 0m0.079s
sys 0m0.019s

ON LINUX UNIX
It is better to use perf then. You can run the command several times. You can also choose among many different categories of statistics. You need to be root.This is an example which runs the program 100 times:

sudo perf stat -e cpu-clock -r50 java HeeloWorld
Hello World! # printed 100x

Performance counter stats for ‘java HeeloWorld’ (100 runs):

190.546723 cpu-clock (msec) # 0.992 CPUs utilized ( +- 0.91% )

0.265467836 seconds time elapsed ( +- 1.49% )

Automatically generate java backend and rest api from existing database. Part 2

Expanding on the previous subject about the generation of a back-end and rest API from existing database .

  • Netbeans has a generate entities from database script
  • Jeddict  . This could be already be pre-installed in some Netbeans products
  • Dali persistence tools from Eclipse
  • Hibernate Tools

Jipster offers the on line tool JDL-studio: there you can visually create tables and relationships. Entities generation from database is only available with an external tool. Beware of some cumbersome issues.

 

Last and most important: your good SQL database may be able to generate the text of the most important queries towards you database. Look at PostgreSql . Their pgAdmin toll has many features like these for generating things.

 

 

Gerrit shortcomings

Yes one of the teams uses Gerrit.

They are completely stuck.

Developers are sensing this tool as a impediment to deliver code and participate in the building of the software.

It has happened that there is always only one person that is reviewing code: the least experienced one! Gerrit Code reviews are just toys  for children , there is no improvement in the code, people is afraid to work with it .

This small team also needs a person dedicated 50% of her time to fixing Gerrit , its installation and its environment.

As a temporary solution we have allowed that the most expert developer makes formal code reviews by passing Gerrit.

Gerrit is the wrong solution for truly agile software development

 

 

Autovalue and Eclipse = success

I needed to add the folder target\…\generated_sources amon the src folders..

That was all what was needed .

Neither enablig maven annotation processor in eclipse project, installing of apt plugin  or the following snippet in pom were needed:

<annotationProcessors> <annotationProcessor>com.google.auto.value.processor.AutoValueProcessor</annotationProcessor>

</annotationProcessors–>

Introduction to Java 8

Are you new to Java?
Are you coming back to programming after a long pause?
Good!
It is the best way to enjoy Java 8 .
Forget all the old Java things and jump directly to Java 8.
I was skeptical about it , but I realize that Java 8 is a great thing.

Here we come to some reading advice:

  • General book about Java SE 8

The best available book is Horstmann – Java SE 8 for the Really Impatient (2014) .

  • About error and exception handling

My advice is to read 7 Error Handling by Michael Feathers in the book “Clean Code A Handbook of Agile Software Craftsmanship”. The author as a voice interview online as well.

Bidirectional traversing with ListIterator

Do you need a bidirectional way to skip through a list?

A rewind – forward behavior?

Traversing the list in one way and then in the opposite direction?

The Java ListIterator in the java.util package is here ready made to your help.

An iterator for lists that allows the programmer to traverse the list in either direction.

The previous() and hasPrevious() methods are available in this class and will make the trick.

You can build a ‘media tape’ behaviour on top of it.

 

It has some loose relationships with the Elevator Algorithm used for seeking a disk in its most simplest implementation strategy, the FCFS, First Come First Served. I used this one before for other purposes than seeking a disk.

ArrayList algorithm complexity, performance and gotchas

This piece of advice is about the Java ArrayList and LinkedList. It is not a insight about a general not specified List algorithm.

ArrayList is a random access data store.

LinkedList is sequential access data store.

There are these very important performance issues to know:

  • Remove: ArrayList remove(E ) is painful slow. Please consider LinkedList if this operation is used often in your application. Also the ArrayList positional remove(index) is painful bad. These operations require constant-time in a LinkedList and linear-time in an ArrayList. (*)
  • Use ArrayList constructor that take a size argument. You can measure by your self the defference when using large lists.
  • List Interface : Iterating over the elements in a list is preferable to indexing through it if the caller does not know the implementation.
  • Remember that you have the method addAll() for bulk operations.
  • Add positional add(index) is painful O(n) on ArrayList and O(1) in LinkedList (*)
  • Set positional set(index) is painful O(n) on LinkedList and O(1) in ArrayList   (*)

(*) Also the outline of the Collections Framework states that LinkedList is better than ArrayList for accessing in the middle of the list.

Also:

Do not do this:

 int a1[] = { -23, 12, 21 };
 
for (int v : a1)
           myarraylist.add(new Integer(v));

prefer this instead:

myarrayList.addAll(Arrays.asList(23,12,21)

Also

Arrays.asList

for translating arrays to strings is obsolete. Use this instead:

System.out.println(Arrays.toString(myArray));

Static Lock (ReentrantLock) and SonarQube destroyed my day

Yes, the idea of letting a module pass through a SonarQube validation throw us in putting ‘static’ on a lot of fields. A static happen to be attached to a ReentrantLock …
And many wasted hours of debugging followed because the applications using this module became crazy and not correct.
This piece of code can destry your application:

private static final Lock = new ReentrantLock();

Benchmark , Performance StringJoiner StringBuffer “+” plus operator

This is a benchmark for performance and behaviour of joining strings with:

  • StringJoiner
  • StringBuffer
  • The + operator

The result is that

StringJoiner is the winner.

StringBuffer and ‘+’ are worse and have the same performance.

package org.sample;
 
import java.util.ArrayList;
import java.util.List;
 
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
 
@Warmup(iterations = 10, time = 1)
@Measurement(iterations = 10, time = 1)
@State(Scope.Benchmark)
@Fork(1)
public class JoinString extends JoinStringBase {
	@Param({ "10000" })
	private int CYCLES;
 
	private final int P = 4;
 
	private List listCounter;
 
	private List listA;
	private List listC;
	private List listD;
	private List listXP;
	private List listJOINER;
 
	@Setup(Level.Trial)
	public void setUp() {
		listCounter = new ArrayList&lt;&gt;(CYCLES + 1);
 
		for (int i = 0; i &lt; CYCLES; i++) {
			listCounter.add("" + i);
		}
 
		listA = new ArrayList&lt;&gt;(CYCLES + 1);
		listC = new ArrayList&lt;&gt;(CYCLES + 1);
		listD = new ArrayList&lt;&gt;(CYCLES + 1);
		listXP = new ArrayList&lt;&gt;(CYCLES + 1);
		listJOINER = new ArrayList&lt;&gt;(CYCLES + 1);
 
		System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", Integer.toString(P));
	}
 
	@Benchmark
	public long useStringBuffer() {
		for (int i = 0; i &lt; CYCLES; i++) {
			listC.add(bufferDateString(TEXT));
		}
		return listC.size();
	}
 
	@Benchmark
	public long plusOperator() {
		for (int i = 0; i &lt; CYCLES; i++) {
			listA.add(plusOperator(TEXT));
		}
		return listA.size();
	}
 
	@Benchmark
	public long useStringJoiner() {
		for (int i = 0; i &lt; CYCLES; i++) {
			listJOINER.add(stringJoiner(TEXT));
		}
		return listJOINER.size();
	}
}
 
 
 
import java.util.StringJoiner;
 
class JoinStringBase {
 
	private static final String TWENTY = "20";
	private static final String POINT = ".";
 
	protected static final String TEXT = "_20ge20";
 
	String stringJoiner(String pScad) {
		StringJoiner sj = new StringJoiner("");
		String day = pScad.substring(1, 3);
		String month = pScad.substring(3, 5);
		String monthInt = MONTHS.valueOf(month).month;
		String yearSuffix = pScad.substring(5, 7);
 
		sj.add(day);
		sj.add(POINT);
		sj.add(monthInt);
		sj.add(POINT);
		sj.add(TWENTY);
		sj.add(yearSuffix);
		return sj.toString();
 
	}
 
	final String bufferDateString(String pScad) {
		StringBuffer buffer = new StringBuffer();
		String day = pScad.substring(1, 3);
		String month = pScad.substring(3, 5);
		String monthInt = MONTHS.valueOf(month).month;
		String yearSuffix = pScad.substring(5, 7);
 
		buffer.append(day);
		buffer.append(POINT);
 
		buffer.append(monthInt);
		buffer.append(POINT);
 
		buffer.append(TWENTY);
		buffer.append(yearSuffix);
		return buffer.toString();
	}
 
	final String plusOperator(String pScad) {
		String day = pScad.substring(1, 3);
		String month = pScad.substring(3, 5);
		String monthInt = MONTHS.valueOf(month).month;
		String yearSuffix = pScad.substring(5, 7);
 
		String scadFormat = day + POINT + monthInt + POINT + TWENTY + yearSuffix;
		return scadFormat;
	}
 
	enum MONTHS {
		ge("01"), fb("02"), mz("03"), ap("04"), mg("05"), gn("06"), lg("07"), ag("08"), st("09"), ot("10"), nv("11"), dc("12");
 
		final String month;
 
		MONTHS(String month) {
			this.month = month;
		}
	}
 
	final String plusOperatorXP(String pScad) {
		return pScad.substring(1, 3) + POINT + MONTHS.valueOf(pScad.substring(3, 5)).month + POINT + TWENTY
				+ pScad.substring(5, 7);
	}
 
	final String bufferDateStringXP(String pScad) {
		StringBuffer buffer = new StringBuffer();
 
		String month = pScad.substring(3, 5);
 
		buffer.append(pScad.substring(1, 3));
		buffer.append(POINT);
 
		buffer.append(MONTHS.valueOf(month).month);
		buffer.append(POINT);
 
		buffer.append(TWENTY);
		buffer.append(pScad.substring(5, 7));
		return buffer.toString();
	}
 
}

 

The results in details:

blogJoinString2

REST and CQRS Command Query Responsibility Segregation

CQRS stands for Command Query Responsibility Segregation.
REST goes well with CQRS when we use the paradigm of “REST without PUT”.
The idea is that we no longer PUT the “new” state of an entity, instead we make our mutations be first class  citizen nouns (rather than verbs), and POST them.

REST without PUT has a side-benefit of separating command and query interfaces (CQRS)  and forces consumers to allow for eventual consistency.
We POST command entities to one endpoint (the “C” of CQRS) and GET a model entity from another endpoint  (the “Q”).

JMH how to tips

JMH is a good tool for benchmarking your code.

The following hints will help you to save time in an otherwise not easy set up.

  • always use state and scope at class level. Use Scope.Thread when uncertainn. Like:
    @State(Scope.Thread)
    public class MyClass {
  • No benchmarks to run message? check the include/exclude regexps.
  • Always use Fork s greater than zero.
  • Methods annotated with @Benchmark should be public.
  • Always return something from a benchmarked method.
  • Do you need to organize many becnhmarks though many classes? Use the command line to select a specific benchmark to run. Like:

Please refer to this  document for more hints.

Automatically generate java backend and rest api from existing database

I am working a lot with databases , hibernate , restful apis in Java SE / java EE with HTML clients.
I am pleased to suggest this workflow that relies on Netbeans automatic generation capabilities.

1. Netbeans generates JPA entities from an existing database.

As it is described at Dzone and at Oracle site.
You have also the option of generating JAXB annotations directly in the JPA entity classes.

2. Netbeans generates then restful api from the entities.

As it is described at NetBeans Help site.

It is really a breeze to come up fast with a Java backend.
I am working now on the HTML / javascript/ jquery client and I will let you know.

Generate JPA entities from database

with Netbeans

https://dzone.com/articles/crud-in-90-seconds

https://docs.oracle.com/cd/E40938_01/doc.74/e40142/dev_persistence.htm#BABCGACD

with Eclipse

Eclipse Neon Help: Generate entities from tables

Create JPA project

with JBoss

http://www.itbh.at/jpa-tools-generate-entities-from-tables-does-not-work-with-h2-database/?lang=en

with MyEclipse

https://www.genuitec.com/products/myeclipse/features/

Jboss tools 6.4 with eclipse

I found the documentation surrounding this to be incomplete and confusing.

You need to keep an eye on

  • which version of Eclipse you are using
  • which version of Java you have
  • which is the site Jboss where the tools reside

Jboss tools for Eclipse Luna and for Jboss EPS 6.x work on java 7 .

Do not use Jboss installation sites url or Eclipse marketplace. Use the install new software in Eclipse instead. Insert this URL “http://download.jboss.org/jbosstools/updates/webtools/luna/”

 

Comparison of Automatic REST API Code Generation Tool

I spent some time trying to find a proper tool that is able to generate tidy Java code for REST resources.

My requirements are:

  • The Java code must be readable and not polluted with a lot of imports that refer to the tool maker .
  • The input should be short and simple.

I stopped by two tools as they were the only ones that would comply with my requirements. They are:

  • Restunited
  • Swaggerhub

Swaggerhub wins.

The reason is that a yaml definition file is used as input to Swaggerhub. Also the output consisted of tidy and simple java classes for the server implementations based on jax-rs.

Here it is a yaml specification file you can begin with at Swaggerhub:

swagger: '2.0'
info:
version: '0'
title: Cars Rental Store
description: Allow clients to loan cars
paths:
    /rentals:
        get:
                summary: Product Types
                description: |
                        bla bla bla.
                parameters:
                        - name: rentals_id
                in: query
                    description: id of.
                    required: true
                    type: number
                    format: double
                tags:
                    - Rentals
                responses:
                    200:
                        description: An array of rentals
                        schema:
                            type: array
                            items:
                                $ref: '#/definitions/Rental'
                        default:
                            description: Unexpected error
                            schema:
                                $ref: '#/definitions/Error'
definitions:
        Car:
                type: object
                properties:
                        car_id:
                            type: number
                            description: Unique identifier representing a specific car.
                        title:
                            type: string
                            description: Description of car.
        Rentals:
                type: object
                properties:
                        rental_id:
                            type: string
                            description: Unique identifier representing a specific rental.
        Cars:
                type: array
                items:
                        $ref: "#/definitions/Car"
                description: bla bla.
        Error:
                type: object
                properties:
                code:
                        type: integer
                        format: int32
                        message:
                                type: string
                fields:
                        type: string

A REST and websocket programmers daily log

We want to share issues and annoyances that made our trip slower.

ThreadLocalRandom

Good to remember that it exists.

JSON Parsing

It is possible to use Jackson with no JAXB as for these examples
https://github.com/jersey/jersey/tree/2.17/examples/json-jackson
and this specification
https://jersey.java.net/nonav/documentation/latest/media.html#json.jackson

MessageBodyProviderNotFoundException

The MessageBodyProviderNotFoundException has to do with JSON deserialization.
It is the sympton and happens when Jersey client deserializea a map or collections when readEntity fails
http://stackoverflow.com/questions/26818492/serialization-and-deserialization-of-map-with-jersey-and-jackson

Test With ObjectMapper

@Test
public void testGetIt() throws Exception {
String responseMsg = target.path(“apipath”).request().get(String.class);
ObjectMapper mapper = new ObjectMapper();
MyList books = mapper.readValue(
responseMsg, MyListAnswer.class);
System.out.println(books);
}

SERVLET, JERSEY and TOMCAT

Jersey version 2 servlet

If you want to use jersey version 2 you need to replace the servlet definition with the following:

<servlet>
<servlet-name>Jersey-Servlet</servlet-name>
<servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
<init-param>
<param-name>jersey.config.server.provider.packages</param-name>
<param-value>com.memorynotfound.rs</param-value>
</init-param>
</servlet>
===============
<init-param>
<param-name>com.sun.jersey.api.json.POJOMappingFeature</param-name>
<param-value>true</param-value>
</init-param>

Cannot call sendError() after the response has been committed

Solution: return after sendError() to avoid chain.doFilter(request, response); to be called

How throttle servlet requests-Solution

Use a  web filter.

ClassNotFoundException: org.glassfish.jersey.servlet.ServletContainer

Solution: .war file is incomplete. Make it again and check that it contains all libraries.

Setting property ‘source’ to ‘org.eclipse.jst.jee.server:appname’ did not find a matching property

You can change the eclipse tomcat server configuration. Open the server view, double click on you server to open server configuration. Then click to activate “Publish module contents to separate XML files”. Finally, restart your server, the message must disappear.

At least one JAR was scanned for TLDs yet contained no TLDs

Set catalina.properties
tomcat.util.scan.StandardJarScanFilter.jarsToSkip=*.jar

Charting, Rickshaw, JavaScript, Servlet, JSON

set “Keep-alive” to … with JAX-RS Response.header() .

IMPLEMENT PUSH WITH WEBSOCKETS IN JAVA

Client javascript parsing of Json

Use JSON.parse in onMessage(), or everything will be caos 🙂

function onMessage(event) {
  var json = JSON.parse(event.data);

Some help

http://stackoverflow.com/questions/25947790/real-time-updates-from-database-using-jsf-java-ee/

MAVEN

Update repository , and mirrors
https://maven.apache.org/guides/mini/guide-configuring-maven.html

Install custom jars
https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html

GIT

Setting up kdiff3 as the Default Merge Tool for git on Windows

Edit .gitconfig , it is usually in your user directory. There may exist other .gitconfig files, pay attention please!

[merge]
tool = kdiff3
[mergetool “kdiff3”]
path = “D:/Progra~1″/KDiff3/kdiff3.exe
cmd = “D:\\Progra~1\\KDiff3\\kdiff3.exe” $BASE $LOCAL $REMOTE -o $MERGED
cmdOK = “D:/Progra~1″/KDiff3/kdiff3.exe
keepBackup = false
trustExitCode = false

 

Apache Kafka crashes, failures, partition reassignment. Trasparency is a requirement for distributed systems

“…your partitions will be reassigned when your app crashes, so you won’t even be able to detect the failure later unless you carefully track it …”

This was about the Kafka simple consumer from here http://engineering.onlive.com/2013/12/12/didnt-use-kafka/

Also from official documentations: “… For manually assigning replicas to…”.

We look for comments and discussion about trasparency in Kafka.

It can be on these issues as well on other ones:

Performance transparency: Client programs should continue to perform satisfactorily while the load on the service varies within a specified range.
A major task of many middleware platforms is to make remote invocations look like local ones.
Is the programmer protected from the details of data representation and marshalling?
A common requirement when data are replicated is for replication transparency. That is, clients should not normally have to be aware that multiple physical copies of data exist. As far as clients are concerned, data are organized as individual logical objects and they identify only one item in each case when they request an operation to be performed. Furthermore, clients expect operations to return only one set of values.

Is this provided by Kafka?

Message loss and Apache Kafka

There exists only one way to avoid message loss: use the tools that banks use.
They loose the money and feel the pain if a message dies and are seriuos about this. If you want kafka with no message loss so you need to :

  • implement your own protection layer around Kafka
  • monitor everything and everywhere.

Monitor is a beast to do operations fast. The SLAs are out of its possibility.

Kafka documentation.
We find that there is a lot of unconsistency. The same concept is exposed more than one time in a different way at different places. Some descriptions are just impossible to understand. LinkedIn guys are good at creating a huge cloud of blogs , benchmarks and assertions that no one can check. They also assert that it fulfils all the three requirements of CAP theorem. And are completely confused where they explain this. It tries to be many things contemporary. No university department has made any research trying to classify and find its nature just because it is a bag containing many things. There is plenty of performance benchmark instead which are done in friendly environments where all the nodes, consumers, zookeeper , producers, sinks stay well and never fail. The benchmarks are done from Linkedin girls or are based on their assertions.