Python – Remove Columns of Duplicate Elements


When it is required to remove columns of duplicate elements, a method is defined that creates an empty set. The list is iterated over, and if it is not found, it is added to the set.

Example

Below is a demonstration of the same

from itertools import chain

def remove_dupes(my_sub):

   my_string = set()
   for i, elem in enumerate(my_sub):

      if elem not in my_string:
         my_string.add(elem)
      else:
         yield i

my_list = [[5, 1, 6, 7, 9], [6, 3, 1, 9, 1], [4, 2, 9, 8, 9], [5, 1, 6, 7, 3]]

print("The list is : ")
print(my_list)

K = 3

temp_idxs = set(chain.from_iterable(remove_dupes(sub) for sub in my_list))

my_result = [[elem for i, elem in enumerate( sub) if i not in temp_idxs] for sub in my_list]

print("The result is : ")
print(my_result)

Output

The list is :
[[5, 1, 6, 7, 9], [6, 3, 1, 9, 1], [4, 2, 9, 8, 9], [5, 1, 6, 7, 3]]
The result is :
[[5, 1, 6, 7], [6, 3, 1, 9], [4, 2, 9, 8], [5, 1, 6, 7]]

Explanation

  • The required packages are imported into the environment.

  • A method named ‘remove_dupes’ is defined that takes a list as a parameter.

  • An empty set is created.

  • The elements in the list are enumerated over, and if an element is not found, it is added to the empty set.

  • Otherwise, the iterator of the enumeration is yielded.

  • Outside the function, a list of list is defined and is displayed on the console.

  • A value for ‘K’ is defined.

  • The method is called by iterating over it.

  • It is converted to a set and is assigned to a variable.

  • Again, it is enumerated, and iterated and stored in a result variable.

  • This result variable is displayed as output on the console.

Updated on: 20-Sep-2021

102 Views

Kickstart Your Career

Get certified by completing the course

Get Started
Advertisements