[OP-139] T#9813 Protect OBM about contact duplication Created: 26/Nov/14  Updated: 29/Jan/15  Resolved: 28/Jan/15

Status: Closed
Project: Opush
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0.6

Type: New Feature Priority: Normal
Reporter: Matthieu EXT_BAECHLER Assignee: Stephane COLSON
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Cloners
is cloned by OP-158 CLONE 3.1 - T#9813 Prevent resent req... Closed
Story Points: 6
Rank: 23
Sprint: OBM Sprint 2015#1

 Description   

Pre-conditions

Use an Android device

Reproduction steps

  1. usera creates a contact on its phone
  2. server is slow enough to have the phone issue the same request
  3. both request eventually get processed

Current behavior

  1. The contact is inserted twice : NOK

Expected behavior

  1. The contact is inserted once : OK

Additional informations

This changeset aims to protect the backend about contact duplication.
That protection is implemented using the hash of every contact created by a device.
When a contact hash is already known by Opush, it'll not send the creation command to its backend.

This fix has been done to handle the never fixed "ClientId" bug of Android, see https://code.google.com/p/android/issues/detail?id=61100



 Comments   
Comment by Jenkins Continuous Integration Server [ 21/Jan/15 ]

ABORTED: Integrated in opush-master-3.0 #24
OP-139 Add an integration test (thilaire: 19eba662803bffffd9f811e0110163e7ca5a4351)

  • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
    OP-139 Add Cassandra dao (thilaire: c3fcc4f3152df556127802055ce4d8530b245c42)
  • push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java
  • push-dao-cassandra/src/main/java/org/obm/push/cassandra/OpushCassandraModule.java
  • push-dao-testsuite/pom.xml
  • push-server/src/main/resources/versions/4/contact_creation.cql
  • push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java
  • push-dao-cassandra/src/main/java/org/obm/push/cassandra/MigrationModule.java
  • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/CassandraStructure.java
  • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java
  • push-dao-cassandra/src/test/java/org/obm/push/cassandra/dao/ContactCreationDaoImplTest.java
    OP-139 Check contact creation is already done (thilaire: c5a5ff29a4753dead9e1f35ae9d206c3c1f72bf3)
  • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java
  • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
  • push-dao-testsuite/pom.xml
  • obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java
  • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendHierarchyChangesTest.java
  • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
  • obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java
Comment by Thomas HILAIRE [ 22/Jan/15 ]

Impacts: Contact creation and modification
QA: Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook.
Verify with a set of device that :

  • you can create the same contact in two different addressbooks
  • you can modify any contact as many times as you wish
  • two or more contacts created with exactly the same informations :
    • only create one in OBM
    • can duplicate contacts on the phone
    • modifying any contact (even duplicated) on the phone forward the modification to OBM
    • drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone
Comment by Stephane COLSON [ 22/Jan/15 ]

I QA this

Comment by Stephane COLSON [ 22/Jan/15 ]
logs warning a duplicate has been avoided
[WARN ]... A creation is discarded as a recent similar creation has been found
Comment by Stephane COLSON [ 22/Jan/15 ]

Tested with OBM 3.1.2~0.alpha0+git20150122-1408-c7f1c45 + Opush 3.0.6~rc1-1 with Fairphone FP1

Duplication reproduced with Opush 3.0.5, and no more reproduced with 3.0.6-rc1 (with slowing down the server or not) BUT

Reproduction steps

  1. As usera with Android, create a contact "First name = Contact", and "Last name = Test"
  2. Wait for a synchronization. If the contact is created in obm-ui, continue
  3. As usera with Android, delete the contact "Contact Test"
  4. Wait for a synchronization. If the contact is deleted in obm-ui, continue
  5. As usera with Android, create exactly the same contact "First name = Contact", and "Last name = Test"
  6. Wait for a synchronization

Current behavior

  1. The contact is never created NOK
  2.  [t224/cSync/r80/usera@obm24.lyn.lng] A creation is discarded as a recent similar creation has been found 

    NOK

Expected behavior

  1. The contact is created OK

Additional informations

According to opush leaders, it is a regression.

Comment by Jenkins Continuous Integration Server [ 26/Jan/15 ]

ABORTED: Integrated in opush-master-3.0 #25
OP-139 A deleted contact can be created again (thilaire: bdcf382a017868e44a80b3ed460bf7accff9ee60)

  • push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java
  • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
  • obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java
  • push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java
  • push-server/src/main/resources/versions/4/contact_creation.cql
  • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java
  • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java
  • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
Comment by Stephane COLSON [ 27/Jan/15 ]

I QA this

Comment by Thomas HILAIRE [ 27/Jan/15 ]

Impacts: Contact creation, modification and deletion
QA: Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook.
Verify with a set of device that :

  • you can create the same contact in two different addressbooks
  • you can modify any contact as many times as you wish
  • two or more contacts created with exactly the same informations :
    • only create one in OBM
    • can duplicate contacts on the phone
    • modifying any contact (even duplicated) on the phone forward the modification to OBM
    • drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone
    • such contacts can be delete from opush, then it can be recreated again
    • such contacts can be delete from another client, then it can be recreated again
Comment by Stephane COLSON [ 27/Jan/15 ]

With version 3.0.6-rc2 (and OBM 3.1.1)

Reproduction steps

  1. As usera with Ipad, create a new contact with
  • firstname = Jacques
  • lastname = Martin
  • company = Michelin
  • Phone number (home) = 01 23 45 67 89
  1. As usera with Ipad, create another contact with
  • firstname = Jacques
  • lastname = Martin
  • company = Goodyear
  • Phone number (home) = 02 45 32 23 60

Current behavior

  1. Only one is retrieved in obm-ui NOK

Expected behavior

  1. Both should be uploaded to ui OK

Additional informations

Company and home phone number are not used to see if it's a duplicate. These 2 informations are the first informations in the contact form on an ipad.
We should probably compare all informations. To be discussed

Comment by Thomas HILAIRE [ 28/Jan/15 ]

Impacts: no change comparing than the previous one
QA: same things + more fields are now taken into account so such creation are now considerate as different:
==
Firstname: Serge
Lastname: Gainsbourg
Spouse: Jane Birkin
==
Firstname: Serge
Lastname: Gainsbourg
Spouse: Bambou
==
Note: can you test with accent or specials chars anywhre ?

Comment by Stephane COLSON [ 29/Jan/15 ]

I QA this

Comment by Stephane COLSON [ 29/Jan/15 ]

OK with Opush 3.0.6~rc3-1, tested on a Wheezy with ipad (iOS 8.1.3), Nexus 7 (Android 4.4.3, 4.4.4 and 5.0.2), Fairphone FP1 (Android 4.2.2) and lovely Windows Phone 8.1 Lumia 920

See OP-163 for a new Issue with WP8
Please note that it is possible to have duplicate contacts on the smartphone easily, but this new behavior prevent from sending it to OBM.
It is also possible to have a duplicate contact in OBM when having 2 different contacts already synchronized, and you modify one in order to be exactly the same as the other. Comparison only works for new creation, not with modification.

Comment by Jenkins Continuous Integration Server [ 29/Jan/15 ]

ABORTED: Integrated in opush-master-3.0 #27
OP-139 Add fields taken into account (thilaire: cbd3d960426bff3282acddcf3ac7d3f54074a4c9)

  • obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java
  • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
Generated at Mon Aug 26 12:13:21 CEST 2019 using JIRA 6.1.1#6155-sha1:7188aeec9a6b57d61ea04c52f235f15f55c105e2.