We have had several clients that want to know about entries that have attributes that contain duplicate values. We have written a tool that will locate the entries with duplicate values.

Pass in any attribute name ( as -A <Attribute to check>) and this program will generate a LDIF file either displaying or removing the values.

Use Entirely at Your Own Risk CISUS.COM nor anyone else is responsible if you use this tool or any information on this site and causes damages to anyone or anything! You are required to read Our Standard Disclaimer

Use Entirely at Your Own Risk#

CISUS.COM nor anyone else is responsible if you use this tool or any information on this site and causes damages to anyone or anything! You are required to read Our Standard Disclaimer

Although we have done some testing, we can not guarantee the program. We generate an LDIF file that you can review for correctness and to make sure this is what you want performed. Check the LDIF file.

If you attempt to remove a naming attribute, it will be commented out with a "#" in the file.

Some of the more interesting options#

The -X value and -R# work as follows:
  • -X Will remove all values (except for naming values) of an attribute
  • -R# Where # is a sequence number of the attribute to remove. If there are two values, entering a one (1) will remove the first value, etc.
Check the output file as there is no guarantees what order values will be returned from the server.
Running without -X or -R may provide a confidence that the values will be returned in the same order each time, but the LDAP RFC 4522Content unavailable! (broken link)https://ldapwiki.com/wiki/images/out.png states "The set of attribute values is unordered. Implementations MUST NOT rely upon the ordering being repeatable."
If the attribute is required, attempts will be made to remove the value as removing the value will work as long as there is at least one remaining value.

Data Output#

All out put is located in the same directory as the FindDupes.jar.
  • output will go to stdout and also to data.ldif
  • Using the -d option will put a lot of debug information into willeke-app-debug.log
The data.ldif file is not overwritten when you run the program repetivlely. This allows you to perform a multiple runs of the program and have all the out put in the same file. Not sure if this is the "best" thing or not, so let us know.

Usage#

The line below will generate the out pur as shown:
java -jar FindDupes.jar  -v  -h 192.x.x.x -p389 -Dcn=admin,ou=administration,dc=willeke,dc=com -w secretpassword -Acn -F (objectclass=inetorgperson) -bou=people,dc=willeke,dc=com


dn: cn=TestMeDelete,ou=people,dc=willeke,dc=com
# cn: TestMeDelete Naming value can not be removed!
cn: OtherName

dn: cn=VBratton,ou=people,dc=willeke,dc=com
cn: Bratton Vivian
# cn: VBratton Naming value can not be removed!

dn: cn=mcenters,ou=people,dc=willeke,dc=com
cn: Centers Melissa A
# cn: mcenters Naming value can not be removed!

dn: cn=Briich Stephanie D,ou=people,dc=willeke,dc=com
# cn: Briich Stephanie D Naming value can not be removed!
cn: Hogge
cn: Beegle

dn: cn=Andreessen Marc,ou=people,dc=willeke,dc=com
cn: Andreessen, Marc
# cn: Andreessen Marc Naming value can not be removed!

Usage#

Usage: java -jar FindDupes.jar [ -Xdv ] [ -R <remove> ] [ -h <host> ] [ -p <port> ] -D <loginDN> [ -w <bindpassword> ] -b <baseDN> -A <Attribute to check> -F <filter> [ -e <keystore> ] [ -Z <encrypted connection> ] 

    -X    Remove ALL Values - Default=false
    -R    Remove value - Default=0
    -h    host name or IP address - Default="localhost"
    -p    host IP port number. - Default=389
    -D    the DN of the object used for authentication - Required Argument
    -w    the password for the DN of the object used for authentication - Default=""
    -b    the baseDN where entries are or the FDN of a single entry - Required Argument
    -A    cn - Required Argument
    -F    RFC-2254 compliant LDAP search filter - Required Argument
    -d    enable API debug output - Default=false
    -e    Path to a Java Keystore.  A valid certificate in the keystore enables
          an encrypted TLS connection.  See also the -Z option. - Default=""
    -v    enable verbose output - Default=false
    -Z    sets the type of encrypted connection.  A Keystore must be specified
            with the -e option to enable an encrypted connection.
            SSL   - Establishes an encrypted connection using 
                    SSL.  The default port is 636
            TLS   - Establishes an encrypted connection using 
                    TLS.  The default port is 389 - Default="TLS"

Download Duplicate Attribute Value Locater Tool#

Duplicate Attribute Value Locater Tool Jar File

Limitations#

FindDupes should work on DIT with up to 500,000 entries. If you have more than that, contact us.

Standard Disclaimer#

Copyright And Intellectual Property Information#

Java Versions And Running Programs#

More Information#

There might be more information for this subject on one of the following: