possible bug with insert operation

[atfreddos]

C:\Program Files\Google\Cloud SDK2>gcloud -v

Google Cloud SDK 136.0.0
app-engine-java 1.9.46
cloud-datastore-emulator 1.2.1
core 2016.11.30
core-win 2016.11.07
gcloud

Catatumbo version 1.1.1
Windows 7
Eclipse Mars,
JRE System Library [JavaSE 1.8] Oracle

Insert problem. The logical requirement for an insert is that the database entity inserted is exactly the same as the entity presented to the database. In the example below, the input fields aRUserId=arcathy, aRLoginName=maskedman have been set to null in the Local Datastore. I have not used all fields in the class.

The problem in steps:

Step 1.
Connect to Local Datastore
INFO: Datastore Environment Host = localhost:8426 ProjectID = ds1-test
INFO: DSConnect: Datastore Connection created.

Step 2.
Create an example class 'userbean1'
INFO: created: UserAccountSessionBean [id=null, userId=Zorro, aRUserId=arcathy, aRLoginName=maskedman, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]

Step 3.
Using Data Access Object, try to save 'userbean1' into Local Datastore
DAO IN > SAVE: userBean1: UserAccountSessionBean [id=null, userId=Zorro, aRUserId=arcathy, aRLoginName=maskedman, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]

Step 3(i) Perform actual insert: (actual method called: 'UserAccountSessionBean sbean = DSConnect.INSTANCE.em().insert( bean );')
Precondition : Inserting object:UserAccountSessionBean [id=null, userId=Zorro, aRUserId=arcathy, aRLoginName=maskedman, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]
call .insert( userbean1 )
Postcondition: Insert returns:UserAccountSessionBean [id=38, userId=Zorro, aRUserId=null, aRLoginName=null, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]

DAO OUT < SAVE: userBean1: UserAccountSessionBean [id=38, userId=Zorro, aRUserId=null, aRLoginName=null, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]

Step 4.
Get (load) saved (inserted) object using Id=38
DATASTORE ENTRY userBean1 by id =
UserAccountSessionBean [id=38, userId=Zorro, aRUserId=null, aRLoginName=null, uniqueId=arcathy-2, uuid=mdmdmdmdmdmdmdmd, firstName=fred, lastName=smith, fullName=fred smith, email=null, admin=false, ip=null, channelId=null, websocketEndpointURI=null, sessionID=null, issuedAt=null, expiration=null, lastActive=null, lastLoginOn=null, lastReported=null, lastPage=null, tK=null]

---

Java Class,
Note that getters and setters have been created by Eclipse.

import java.io.Serializable;
import java.util.logging.Logger;

@entity(kind="usersessions")
public class UserAccountSessionBean implements Serializable {

private static final long   serialVersionUID = -2387288350780145593L;
private static final Logger log              = Logger.getLogger(UserAccountSessionBean.class.getName());

@Identifier
public Long id;    

@Property(indexed = true)
private String userId      = null;    

private String aRUserId;    
private String aRLoginName = null;    
private String uniqueId    = null;    

@Property(indexed = true)
private String uuid        = null;    

private String firstName = null;    
private String lastName  = null;    
private String fullName  = null;    

private String email = null;    


@Property(indexed = true)
private boolean admin = false;    

private String ip = null;    

private String channelId = null;

private String websocketEndpointURI = null;

@Property(indexed = true)
private String sessionID    = null;        
private Long   issuedAt     = null;    
private Long   expiration   = null;    
private Date   lastActive   = null;
private Date   lastLoginOn  = null;
private Date   lastReported = null;
private String lastPage     = null;    
private String tK           = null;    


// ======================================================================
// Two Constructors
// ======================================================================


// no arg constructor
public UserAccountSessionBean() {
    super();
}

public UserAccountSessionBean(String loginArId, Integer loginProvider) {
    this();
    this.setUniqueId(loginArId + "-" + loginProvider);
    this.setaRUserId(loginArId);
}

// ======================================================================
// Set Some Basic Info
// ======================================================================

public void setBasicInfo(String screenName, String emailAddress, String uniqueId) {
    this.fullName = screenName;
    this.email    = emailAddress;
    this.uniqueId = uniqueId;
}


// ======================================================================
// toString()
// ======================================================================
@Override
public String toString() {
	return "UserAccountSessionBean [id=" + id + ", userId=" + userId + ", aRUserId=" + aRUserId + ", aRLoginName="
			+ aRLoginName + ", uniqueId=" + uniqueId + ", uuid=" + uuid + ", firstName=" + firstName + ", lastName="
			+ lastName + ", fullName=" + fullName + ", email=" + email + ", admin=" + admin + ", ip=" + ip
			+ ", channelId=" + channelId + ", websocketEndpointURI=" + websocketEndpointURI + ", sessionID="
			+ sessionID + ", issuedAt=" + issuedAt + ", expiration=" + expiration + ", lastActive=" + lastActive
			+ ", lastLoginOn=" + lastLoginOn + ", lastReported=" + lastReported + ", lastPage=" + lastPage + ", tK="
			+ tK + "]";
}


// ======================================================================
// Getters and Setters
// ======================================================================

 public void setId(Long id) {
	this.id = id;
 }


public String getWebsocketEndpointURI() {
    return websocketEndpointURI;
}

public void setWebsocketEndpointURI(String websocketEndpointURI) {
    this.websocketEndpointURI = websocketEndpointURI;
}

public void setUserId(String userId) {
    this.userId = userId;
}

public void setaRUserId(String aRUserId) {
    this.aRUserId = aRUserId;
}

public void setaRLoginName(String aRLoginName) {
    this.aRLoginName = aRLoginName;
}

public void setUniqueId(String uniqueId) {
    this.uniqueId = uniqueId;
}

public void setFirstName(String firstName) {
    this.firstName = firstName;
}

public void setLastName(String lastName) {
    this.lastName = lastName;
}

public void setFullName(String fullName) {
    this.fullName = fullName;
}

public void setEmailAddress(String email) {
    this.email = email;
}

public void setAdmin(boolean admin) {
    this.admin = admin;
}

public void setIp(String ip) {
    this.ip = ip;
}

public void setChannelId(String channelId) {
    this.channelId = channelId;
}

public void setIssuedAt(Long issuedAt) {
    this.issuedAt = issuedAt;
}

public void setExpiration(Long expiration) {
    this.expiration = expiration;
}

public void setLastActive(Date lastActive) {
    this.lastActive = lastActive;
}

public void setLastLoginOn(Date lastLoginOn) {
    this.lastLoginOn = lastLoginOn;
}

public void setLastReported(Date lastReported) {
    this.lastReported = lastReported;
}

public void setLastPage(String lastPage) {
    this.lastPage = lastPage;
}

public void settK(String tK) {
    this.tK = tK;
}

public static Logger getLog() {
    return log;
}

public String getUserId() {
    return userId;
}

public String getaRUserId() {
    return aRUserId;
}

public String getaRLoginName() {
    return aRLoginName;
}

public String getUniqueId() {
    return uniqueId;
}

public String getFirstName() {
    return firstName;
}

public String getLastName() {
    return lastName;
}

public String getFullName() {
    return fullName;
}

public String getEmailAddress() {
    return email;
}

public boolean isAdmin() {
    return admin;
}

public String getIp() {
    return ip;
}

public String getChannelId() {
    return channelId;
}

public Long getIssuedAt() {
    return issuedAt;
}

public Long getExpiration() {
    return expiration;
}

public Date getLastActive() {
   return  lastActive;
}

public Date getLastLoginOn() {
    return  lastLoginOn;
}

public Date getLastReported() {
    return  lastReported;
    
}

public String getLastPage() {
    return lastPage;
}

public String gettK() {
    return tK;
}

public Long getId() {
    return id;
}

public String getEmail() {
    return email;
}

public String getSessionID() {
    return sessionID;
}

public void setEmail(String email) {
    this.email = email;
}

public void setSessionID(String sessionID) {
    this.sessionID = sessionID;
}

public String getUuid() {
    return uuid;
}

public void setUuid(String uuid) {
    this.uuid = uuid;
}

} // end class

[sai-pullabhotla]

This is a very interesting issue. Part of it is because of the field names and how IDEs are generating the setter/getter methods.

If a field name is xY, one (at least me) would normally expect that the getter method would be getXY() and the setter method to be setXY(). However, IDEs are treating this as a special case and creating setxY and getxY methods. Yes, I tested this on Eclipse, NetBeans and IDEA and all are behaving the same way.

Catatumbo is looking for setXY and getXY and since those methods do not exist, and the fact that the field does not have explicit @Property annotation, it ignores the field from persistence.

Same applies to your field - aRUserId. Catatumbo expects the accessor methods to be setARUserId and getARUserId. But the IDEs are generating setaRUserId and getaRUserId, so this field is skipped from persistence.

Java's own API (JavaBeans) - PropertyDescriptor.java uses the below algorithm to generate getter and setter, which does not comply with the IDE's generated methods. The converted name is then prepended with set/get/is. This is exactly what Catatumbo does too in the IntrospectionUtils.getCapitalizedName() method.

Code from Java's PropertyDescriptor

    /**
     * Returns a String which capitalizes the first letter of the string.
     */
    public static String capitalize(String name) {
        if (name == null || name.length() == 0) {
            return name;
        }
        return name.substring(0, 1).toUpperCase(ENGLISH) + name.substring(1);
    }

Now, the JavaBeans Introspector works differently and matches with the IDE's behavior.

Sample Code

package com.example.java8;

import java.beans.BeanInfo;
import java.beans.IntrospectionException;
import java.beans.Introspector;
import java.beans.PropertyDescriptor;

/**
 * @author Sai Pullabhotla
 *
 */
public class Test {

	private int age;

	private int xIndex;

	public int getAge() {
		return age;
	}

	public void setAge(int age) {
		this.age = age;
	}

        //Note how IDEs have generated the getter and setter :) 
	public int getxIndex() {
		return xIndex;
	}

	public void setxIndex(int xIndex) {
		this.xIndex = xIndex;
	}

	public static void main(String[] args) throws IntrospectionException {
		BeanInfo beanInfo = Introspector.getBeanInfo(Test.class);
		PropertyDescriptor[] properties = beanInfo.getPropertyDescriptors();
		for (PropertyDescriptor property : properties) {
			System.out.println(property.getName() + "--->" + property.getReadMethod());
			System.out.println(property.getName() + "--->" + property.getWriteMethod());
		}

		// Uncommenting the xIndex would throw an exception because the method
		// lookup fails.
		String[] fields = { "age", /* "xIndex" */ };
		for (String field : fields) {
			PropertyDescriptor pd = new PropertyDescriptor(field, Test.class);
			System.out.println(pd.getReadMethod());
			System.out.println(pd.getWriteMethod());
		}
	}

}

Output

age--->public int com.example.java8.Test.getAge()
age--->public void com.example.java8.Test.setAge(int)
xIndex--->public int com.example.java8.Test.getxIndex()
xIndex--->public void com.example.java8.Test.setxIndex(int)
public int com.example.java8.Test.getAge()
public void com.example.java8.Test.setAge(int)

The threads below are worth taking a look at:

http://stackoverflow.com/questions/2948083/naming-convention-for-getters-setters-in-java

http://stackoverflow.com/questions/15115072/how-to-identify-setter-method-using-property-name

That said - I do feel like Catatumbo needs to be fixed to be compatible with the IDEs, so I will go ahead and provide a fix.

[sai-pullabhotla]

Thinking more on this - I'm not sure what is really correct. I feel like the JavaBeans framework itself is broke based on the sample code I've posted. PropertyDescriptor expects a different method than what the Introspector finds. Any of the watchers have any thoughts to share??

[sai-pullabhotla]

The fix (to be compatible with IDE generated accessor methods) would be changing the com.jmethods.catatumbo.impl.IntrospectionUtils#getCapitalizedName method as follows. The if block is new.

	/**
	 * Capitalizes the given field name.
	 *
	 * @param fieldName
	 *            the field name
	 * @return capitalized field name.
	 */
	public static String getCapitalizedName(String fieldName) {
		if (fieldName.length() > 1 && Character.isUpperCase(fieldName.charAt(1))) {
			return fieldName;
		}
		return Character.toUpperCase(fieldName.charAt(0)) + fieldName.substring(1);
	}

I will check in this code soon with test cases, if I don't get any further feedback.

@atfreddos - can you possibly clone the repo and make the above change and see if your entities get mapped correctly?

[atfreddos]

Hi Sai, ok will do shortly. I found a discussion of the issue here, it may be of help or interest:
https://dertompson.com/2013/04/29/java-bean-getterssetters/

[atfreddos]

Hi Sai,
I cloned the repo, made the fix, made the jar and it worked in Eclipse as expected. Well done, thanks for speedy response. I will continue testing.
Thanks.

[sai-pullabhotla]

@atfreddos - Version 1.1.3 should be available in Maven Central in a few hours. This release contains the fix for this issue. Thanks for reporting!