CSV Synchronization Project with duplicate entries in Import file

You haven't specified the version you are using so I assume it is not 8.1 or higher.

With version 8.1 we introduced the ability to specify the behavior of the CSV connector for duplicate keys.

Hi there,

I have version 8.1.3.

I was not aware of this option. The problem is that I need to read the two registers because I have to save values of  the two registers. It's possible?

A grreet.

If your only key is the ProfitCenterID, I am unable to see why you need to take both lines and not just the last one.

And no, it is not possible to handle the duplicate items, as either INSERT or UPDATE.

But where do you store information then when you want to have it for the same employee?

I explain.

I have a SAP HR of employees. I can have a employee what he have two records in SAP  because he works for tow societies of the same company. Now, SAP send a CSV with the information of the employees and this employee have two records. If I want have a identity in One Identity for this employee, how can save the information of the two records?

I think you need to decide what these SAP records should represent in your system. Should they be handled as sub-identities? In that case, each ProfitCenterID / LocalityName combination would be the key to identify the subidentities. Or, if you want to have just to location entries for the same employee, you need to import the location assignments to the M:N table PersonInLocality instead of the primary location assignment via Person.UID_Locality.

It is your choice.

Pardon, the example can be confusing.

I have the fields Nºpers what represents the records of SAP and Employee identifier (NIF/NIE in my case).

As you can see, there are records with the same Nif / Nie. Would you have to create that M: N table?

I want the field "Central Account" of One Identity is the field Nif/Nie and  can save the two values  of the field "Nºpers".

A greets.

Each combination of personnell number and Nif/Nie would be a different person for me. So the key for the file would be the combination of both fields.

I need that is the same person.

Could I create a extended table relationed with Person? In this table,  Each combination of personnell number and Nif/Nie would be a different register and then, create a custom process for I create the Person or update if it exist

As you see?

Yes, you can create additional tables and relations to the person table if that is what you want. I do not know if that is the right choice here but, again this is up to you and your use-case.

Hello Markus,

I have a similar issue. I would like to have the primary key of my CSV file as the combinatioin of two fields. If I mark both fields as key columns, the connector has a weird behavior. For example I have this CSV:

The expected result is that:
- Access Right 1 is inserted as group of target system Application 1
- Access Right 1 is inserted as group of target system Application 2
- Access Right 2 is inserted as group of target system Application 1
The last row must be ignored. 

In summary, I would like that the combination of field Application-Access Right is the primary key.

Instead, I get that the connector reads just the first row. It seems like both fields are considered to be unique and not just the combination of them.

The version of OneIM I'm using is the 9.1.

Can you help me?
Thank you in advance,

Lucrezia

Hello Markus,

I have a similar issue. I would like to have the primary key of my CSV file as the combinatioin of two fields. If I mark both fields as key columns, the connector has a weird behavior. For example I have this CSV:

The expected result is that:
- Access Right 1 is inserted as group of target system Application 1
- Access Right 1 is inserted as group of target system Application 2
- Access Right 2 is inserted as group of target system Application 1
The last row must be ignored. 

In summary, I would like that the combination of field Application-Access Right is the primary key.

Instead, I get that the connector reads just the first row. It seems like both fields are considered to be unique and not just the combination of them.

The version of OneIM I'm using is the 9.1.

Can you help me?
Thank you in advance,

Lucrezia

Hi Lucrezia,

the solution in your case is to not define a Key for your CSV file. 

In your mapping, create a virtual property for the CSV that combines both columns and use that virtual property as matching rule.

Hello Markus,

thanks fo your answer. In your way how is it possible to maintain the "Take first" behavior for duplicate management? I have to set the key in the CSV file for this, isn't it?

As the mentioned setting is used if duplicate keys are detected, the setting will not be effective.

So in summary, it is not possible to reach our desired behavior, am I right? 

We aren't able to create a key AccessRight + Application, so that we can exploit the duplicate keys management?

At least not with the synchronization engine as is assuming that either adding a key column to the csv or removing the duplicate entries are no option. One other option would be to use other means of importing the data like using data importer with a customized script or some other means of removing the duplicates  from the file (in a process step perhaps) before processing the CSV file.