Touchpoint completed state got automatically deleted in HCL Connections 6.5.1 - ibm-connections

Question

After enabling HCL Touchpoint on HCL Connections 6.5.1. According to the default configuration, we have setTimeDuration set in /mnt/opt/IBM/WebSphere/AppServer/profiles/CnxNode01/installedApps/CnxCell/Touchpoint.ear/touchpoint.war/js/startup.js. So it should re-appear only after 6 months. But on my test users, it re-appears just after ~1 hour.
Analyzing the problem: Deleted completed state
To debug/analyze this, I found out that touchpoint stores its data in the PEOPLEDB database, which contains a table EMPINST.PROFILE_EXTENSIONS. It uses PROF_PROPERTY_ID = 'touchpointState' to store the timestamp when touchpoint is completed (= a user confirms all steps). In this case, PROF_VALUE contains JSON like {"state":"complete","timestamp":1599763075000} which means the user completed it on 2020-09-10.
I created the following query to get the name, timestamp and the date in human readable form from the completed users:
SELECT e.PROF_DISPLAY_NAME, ext.PROF_VALUE, replace(REPLACE(ext.PROF_VALUE, '}', ''), '{"state":"complete","timestamp":', '') AS timestamp,
date((((replace(REPLACE(ext.PROF_VALUE, '}', ''), '{"state":"complete","timestamp":', '') / 1000)-5*3600)/86400)+719163) AS date
/*SELECT count(*)*/
FROM EMPINST.PROFILE_EXTENSIONS ext
LEFT JOIN EMPINST.EMPLOYEE e ON (e.PROF_KEY=ext.PROF_KEY)
WHERE PROF_PROPERTY_ID = 'touchpointState'
ORDER BY replace(REPLACE(ext.PROF_VALUE, '}', ''), '{"state":"complete","timestamp":', '') desc
Example result:
While this seemed to work, I re-run this query some time later (about 1h) and all those new rows were gone! They got deleted from the database. As a result, the users are redirected to touchpoint again and have to complete it a second time.
I don't know why they got deleted and how we can stop it. On the first run they were deleted after one admin user completed touchpoint. But later also after normal users ran them.

Answer 1

The problem was, that those attributes were left in ${tdisol}/TDI/conf/LotusConnections-config/tdi-profiles-config.xml:
<simpleAttribute extensionId="recommendedTags" length="256" sourceKey="recommendedTags" />
<simpleAttribute extensionId="departmentKey" length="256" sourceKey="departmentKey" />
<simpleAttribute extensionId="privacyAndGuidelines" length="256" sourceKey="privacyAndGuidelines" />
<simpleAttribute extensionId="touchpointState" length="256" sourceKey="touchpointState" />
<richtextAttribute extensionId="touchpointSession" maxBytes="1000000" sourceKey="touchpointSession" />
Just comment them out with <!-- and -->.
It's also required to remove them from ${tdisol}/TDI/conf/LotusConnections-config/profile-types.xml like this:
<!--
<property>
<ref>recommendedTags</ref>
<updatability>readwrite</updatability>
<hidden>true</hidden>
<fullTextIndexed>false</fullTextIndexed>
</property>
<property>
<ref>departmentKey</ref>
<updatability>read</updatability>
<hidden>true</hidden>
<fullTextIndexed>true</fullTextIndexed>
</property>
<property>
<ref>privacyAndGuidelines</ref>
<updatability>readwrite</updatability>
<hidden>true</hidden>
<fullTextIndexed>false</fullTextIndexed>
</property>
<property>
<ref>touchpointState</ref>
<updatability>readwrite</updatability>
<hidden>true</hidden>
<fullTextIndexed>false</fullTextIndexed>
</property>
<property>
<ref>touchpointSession</ref>
<updatability>readwrite</updatability>
<hidden>true</hidden>
<fullTextIndexed>false</fullTextIndexed>
</property>
-->
Even those attributes were set in the default tdisol configuration from HCL, it's wrong because this would lead TDI to overwrite ALL three touchpoint attributes from the LDAP. Usually those fields were not present in the domain. In this case, touchpoint DELETES all attributes - on every run.
Since our TDI was scheduled to run every 30 minutes, it deleted ALL TP related information on every run.
As an alternative, you could also exclude all listed properties above in map_dbrepos_from_source.properties like this:
extattr.privacyAndGuideLines=null
extattr.touchpointState=null
...