Opush
  1. Opush
  2. OP-139

T#9813 Protect OBM about contact duplication

    Details

    • Type: New Feature New Feature
    • Status: Closed Closed
    • Priority: Normal Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.6
    • Labels:
      None
    • Story Points:
      6
    • Rank:
      23
    • Sprint:
      OBM Sprint 2015#1

      Description

      Pre-conditions

      Use an Android device

      Reproduction steps

      1. usera creates a contact on its phone
      2. server is slow enough to have the phone issue the same request
      3. both request eventually get processed

      Current behavior

      1. The contact is inserted twice : NOK

      Expected behavior

      1. The contact is inserted once : OK

      Additional informations

      This changeset aims to protect the backend about contact duplication.
      That protection is implemented using the hash of every contact created by a device.
      When a contact hash is already known by Opush, it'll not send the creation command to its backend.

      This fix has been done to handle the never fixed "ClientId" bug of Android, see https://code.google.com/p/android/issues/detail?id=61100

        Issue Links

          Activity

          Hide
          Jenkins Continuous Integration Server added a comment -

          ABORTED: Integrated in opush-master-3.0 #24
          OP-139 Add an integration test (thilaire: 19eba662803bffffd9f811e0110163e7ca5a4351)

          • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
            OP-139 Add Cassandra dao (thilaire: c3fcc4f3152df556127802055ce4d8530b245c42)
          • push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java
          • push-dao-cassandra/src/main/java/org/obm/push/cassandra/OpushCassandraModule.java
          • push-dao-testsuite/pom.xml
          • push-server/src/main/resources/versions/4/contact_creation.cql
          • push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java
          • push-dao-cassandra/src/main/java/org/obm/push/cassandra/MigrationModule.java
          • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/CassandraStructure.java
          • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java
          • push-dao-cassandra/src/test/java/org/obm/push/cassandra/dao/ContactCreationDaoImplTest.java
            OP-139 Check contact creation is already done (thilaire: c5a5ff29a4753dead9e1f35ae9d206c3c1f72bf3)
          • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java
          • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
          • push-dao-testsuite/pom.xml
          • obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java
          • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendHierarchyChangesTest.java
          • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
          • obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java
          Show
          Jenkins Continuous Integration Server added a comment - ABORTED: Integrated in opush-master-3.0 #24 OP-139 Add an integration test (thilaire: 19eba662803bffffd9f811e0110163e7ca5a4351 ) integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java OP-139 Add Cassandra dao (thilaire: c3fcc4f3152df556127802055ce4d8530b245c42 ) push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java push-dao-cassandra/src/main/java/org/obm/push/cassandra/OpushCassandraModule.java push-dao-testsuite/pom.xml push-server/src/main/resources/versions/4/contact_creation.cql push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java push-dao-cassandra/src/main/java/org/obm/push/cassandra/MigrationModule.java push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/CassandraStructure.java push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java push-dao-cassandra/src/test/java/org/obm/push/cassandra/dao/ContactCreationDaoImplTest.java OP-139 Check contact creation is already done (thilaire: c5a5ff29a4753dead9e1f35ae9d206c3c1f72bf3 ) obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java push-dao-testsuite/pom.xml obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendHierarchyChangesTest.java obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java
          Hide
          Thomas HILAIRE added a comment -

          Impacts: Contact creation and modification
          QA: Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook.
          Verify with a set of device that :

          • you can create the same contact in two different addressbooks
          • you can modify any contact as many times as you wish
          • two or more contacts created with exactly the same informations :
            • only create one in OBM
            • can duplicate contacts on the phone
            • modifying any contact (even duplicated) on the phone forward the modification to OBM
            • drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone
          Show
          Thomas HILAIRE added a comment - Impacts : Contact creation and modification QA : Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook. Verify with a set of device that : you can create the same contact in two different addressbooks you can modify any contact as many times as you wish two or more contacts created with exactly the same informations : only create one in OBM can duplicate contacts on the phone modifying any contact (even duplicated) on the phone forward the modification to OBM drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone
          Hide
          Stephane COLSON added a comment -

          I QA this

          Show
          Stephane COLSON added a comment - I QA this
          Hide
          Stephane COLSON added a comment - - edited
          logs warning a duplicate has been avoided
          [WARN ]... A creation is discarded as a recent similar creation has been found
          Show
          Stephane COLSON added a comment - - edited logs warning a duplicate has been avoided [WARN ]... A creation is discarded as a recent similar creation has been found
          Hide
          Stephane COLSON added a comment -

          Tested with OBM 3.1.2~0.alpha0+git20150122-1408-c7f1c45 + Opush 3.0.6~rc1-1 with Fairphone FP1

          Duplication reproduced with Opush 3.0.5, and no more reproduced with 3.0.6-rc1 (with slowing down the server or not) BUT

          Reproduction steps

          1. As usera with Android, create a contact "First name = Contact", and "Last name = Test"
          2. Wait for a synchronization. If the contact is created in obm-ui, continue
          3. As usera with Android, delete the contact "Contact Test"
          4. Wait for a synchronization. If the contact is deleted in obm-ui, continue
          5. As usera with Android, create exactly the same contact "First name = Contact", and "Last name = Test"
          6. Wait for a synchronization

          Current behavior

          1. The contact is never created NOK
          2.  [t224/cSync/r80/usera@obm24.lyn.lng] A creation is discarded as a recent similar creation has been found 

            NOK

          Expected behavior

          1. The contact is created OK

          Additional informations

          According to opush leaders, it is a regression.

          Show
          Stephane COLSON added a comment - Tested with OBM 3.1.2~0.alpha0+git20150122-1408-c7f1c45 + Opush 3.0.6~rc1-1 with Fairphone FP1 Duplication reproduced with Opush 3.0.5, and no more reproduced with 3.0.6-rc1 (with slowing down the server or not) BUT Reproduction steps As usera with Android, create a contact "First name = Contact", and "Last name = Test" Wait for a synchronization. If the contact is created in obm-ui, continue As usera with Android, delete the contact "Contact Test" Wait for a synchronization. If the contact is deleted in obm-ui, continue As usera with Android, create exactly the same contact "First name = Contact", and "Last name = Test" Wait for a synchronization Current behavior The contact is never created NOK [t224/cSync/r80/usera@obm24.lyn.lng] A creation is discarded as a recent similar creation has been found NOK Expected behavior The contact is created OK Additional informations According to opush leaders, it is a regression.
          Hide
          Jenkins Continuous Integration Server added a comment -

          ABORTED: Integrated in opush-master-3.0 #25
          OP-139 A deleted contact can be created again (thilaire: bdcf382a017868e44a80b3ed460bf7accff9ee60)

          • push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java
          • integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java
          • obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java
          • push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java
          • push-server/src/main/resources/versions/4/contact_creation.cql
          • push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java
          • obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java
          • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
          Show
          Jenkins Continuous Integration Server added a comment - ABORTED: Integrated in opush-master-3.0 #25 OP-139 A deleted contact can be created again (thilaire: bdcf382a017868e44a80b3ed460bf7accff9ee60 ) push-dao-testsuite/src/main/java/org/obm/push/dao/testsuite/ContactCreationDaoTest.java integration-testing/opush/src/test/java/org/obm/opush/command/sync/SyncHandlerOnContactsTest.java obm-backend/src/main/java/org/obm/push/contacts/ContactsBackend.java push-dao/src/main/java/org/obm/push/store/ContactCreationDao.java push-server/src/main/resources/versions/4/contact_creation.cql push-dao-cassandra/src/main/java/org/obm/push/cassandra/dao/ContactCreationDaoImpl.java obm-backend/src/test/java/org/obm/push/contacts/ContactsBackendTest.java obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
          Hide
          Stephane COLSON added a comment -

          I QA this

          Show
          Stephane COLSON added a comment - I QA this
          Hide
          Thomas HILAIRE added a comment -

          Impacts: Contact creation, modification and deletion
          QA: Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook.
          Verify with a set of device that :

          • you can create the same contact in two different addressbooks
          • you can modify any contact as many times as you wish
          • two or more contacts created with exactly the same informations :
            • only create one in OBM
            • can duplicate contacts on the phone
            • modifying any contact (even duplicated) on the phone forward the modification to OBM
            • drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone
            • such contacts can be delete from opush, then it can be recreated again
            • such contacts can be delete from another client, then it can be recreated again
          Show
          Thomas HILAIRE added a comment - Impacts : Contact creation, modification and deletion QA : Opush now remember created contacts for 10 days and does not allow to create twice the same contact for the tuple User/AddressBook. Verify with a set of device that : you can create the same contact in two different addressbooks you can modify any contact as many times as you wish two or more contacts created with exactly the same informations : only create one in OBM can duplicate contacts on the phone modifying any contact (even duplicated) on the phone forward the modification to OBM drop the device link from obm-ui or recreate the account on the phone remove duplications on the phone such contacts can be delete from opush, then it can be recreated again such contacts can be delete from another client, then it can be recreated again
          Hide
          Stephane COLSON added a comment -

          With version 3.0.6-rc2 (and OBM 3.1.1)

          Reproduction steps

          1. As usera with Ipad, create a new contact with
          • firstname = Jacques
          • lastname = Martin
          • company = Michelin
          • Phone number (home) = 01 23 45 67 89
          1. As usera with Ipad, create another contact with
          • firstname = Jacques
          • lastname = Martin
          • company = Goodyear
          • Phone number (home) = 02 45 32 23 60

          Current behavior

          1. Only one is retrieved in obm-ui NOK

          Expected behavior

          1. Both should be uploaded to ui OK

          Additional informations

          Company and home phone number are not used to see if it's a duplicate. These 2 informations are the first informations in the contact form on an ipad.
          We should probably compare all informations. To be discussed

          Show
          Stephane COLSON added a comment - With version 3.0.6-rc2 (and OBM 3.1.1) Reproduction steps As usera with Ipad, create a new contact with firstname = Jacques lastname = Martin company = Michelin Phone number (home) = 01 23 45 67 89 As usera with Ipad, create another contact with firstname = Jacques lastname = Martin company = Goodyear Phone number (home) = 02 45 32 23 60 Current behavior Only one is retrieved in obm-ui NOK Expected behavior Both should be uploaded to ui OK Additional informations Company and home phone number are not used to see if it's a duplicate. These 2 informations are the first informations in the contact form on an ipad. We should probably compare all informations. To be discussed
          Hide
          Thomas HILAIRE added a comment - - edited

          Impacts: no change comparing than the previous one
          QA: same things + more fields are now taken into account so such creation are now considerate as different:
          ==
          Firstname: Serge
          Lastname: Gainsbourg
          Spouse: Jane Birkin
          ==
          Firstname: Serge
          Lastname: Gainsbourg
          Spouse: Bambou
          ==
          Note: can you test with accent or specials chars anywhre ?

          Show
          Thomas HILAIRE added a comment - - edited Impacts : no change comparing than the previous one QA : same things + more fields are now taken into account so such creation are now considerate as different: == Firstname: Serge Lastname: Gainsbourg Spouse: Jane Birkin == Firstname: Serge Lastname: Gainsbourg Spouse: Bambou == Note : can you test with accent or specials chars anywhre ?
          Hide
          Stephane COLSON added a comment -

          I QA this

          Show
          Stephane COLSON added a comment - I QA this
          Hide
          Stephane COLSON added a comment -

          OK with Opush 3.0.6~rc3-1, tested on a Wheezy with ipad (iOS 8.1.3), Nexus 7 (Android 4.4.3, 4.4.4 and 5.0.2), Fairphone FP1 (Android 4.2.2) and lovely Windows Phone 8.1 Lumia 920

          See OP-163 for a new Issue with WP8
          Please note that it is possible to have duplicate contacts on the smartphone easily, but this new behavior prevent from sending it to OBM.
          It is also possible to have a duplicate contact in OBM when having 2 different contacts already synchronized, and you modify one in order to be exactly the same as the other. Comparison only works for new creation, not with modification.

          Show
          Stephane COLSON added a comment - OK with Opush 3.0.6~rc3-1, tested on a Wheezy with ipad (iOS 8.1.3), Nexus 7 (Android 4.4.3, 4.4.4 and 5.0.2), Fairphone FP1 (Android 4.2.2) and lovely Windows Phone 8.1 Lumia 920 See OP-163 for a new Issue with WP8 Please note that it is possible to have duplicate contacts on the smartphone easily, but this new behavior prevent from sending it to OBM. It is also possible to have a duplicate contact in OBM when having 2 different contacts already synchronized, and you modify one in order to be exactly the same as the other. Comparison only works for new creation, not with modification.
          Hide
          Jenkins Continuous Integration Server added a comment -

          ABORTED: Integrated in opush-master-3.0 #27
          OP-139 Add fields taken into account (thilaire: cbd3d960426bff3282acddcf3ac7d3f54074a4c9)

          • obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java
          • obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java
          Show
          Jenkins Continuous Integration Server added a comment - ABORTED: Integrated in opush-master-3.0 #27 OP-139 Add fields taken into account (thilaire: cbd3d960426bff3282acddcf3ac7d3f54074a4c9 ) obm-backend/src/test/java/org/obm/push/contacts/ContactCreationIdempotenceServiceTest.java obm-backend/src/main/java/org/obm/push/contacts/ContactCreationIdempotenceService.java

            People

            • Assignee:
              Stephane COLSON
              Reporter:
              Matthieu EXT_BAECHLER
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile