Python - Join Lists

Working with lists is one of the fundamental tasks in Python programming. Often, there’s a need to combine multiple lists into one.

There are several ways to merge lists in Python.

The + Operator

The simplest and most intuitive way to merge lists is by using the + operator. It creates a new list containing the elements of all the original lists.

📌 Example:

The + operator combines the lists in the order they are provided. The original lists list1 and list2 remain unchanged, and the result is stored in a new variable result.

Drawbacks: A new list is created, which can be inefficient when working with large datasets.

The .extend() Method

If you want to add elements from one list to another "in place" (without creating a new list), use the .extend() method. It modifies the original list directly.

📌 Example:

The .extend() method appends all elements from list2 to the end of list1. While list2 remains unchanged, list1 is modified.

The += operator works similarly to .extend() and is a shorthand version of it.

The * Operator for Unpacking

The * operator allows you to "unpack" lists and combine them into a new list using unpacking syntax.

📌 Example:

Here, *list1 and *list2 extract all elements from the lists, which are then combined into a new list result.

The .append() Method for Nested Lists

If you want to combine lists as nested structures (a list of lists), you can use the .append() method.

📌 Example:

The .append() method adds each list as a single element to result, creating a list of lists.

• Advantages: Useful when you need to preserve nested structure.
• Drawbacks: Does not create a flat list of individual elements.

Using itertools.chain

For more advanced users, the itertools.chain module offers an efficient way to merge lists into an iterable object. To get a list, you need to convert the result using list().

📌 Example:

chain combines the elements of the lists into a single iterable object, which is then converted into a list.

• Advantages: High performance when working with large lists.
• Drawbacks: Requires importing a module and may be overkill for simple tasks.

Difference Between List Addition and itertools.chain

In Python, both itertools.chain and list addition (list1 + list2) can be used to merge lists, but they have key differences.

Advantages of itertools.chain

1. Memory Efficiency

   • list1 + list2 creates a new list, requiring memory allocation for all elements at once. This can be costly for large lists.
   • chain works as an iterator and doesn’t create a copy of the data in memory. This is especially useful when processing data element by element rather than storing it all at once.

from itertools import chain

list1 = [1, 2, 3]
list2 = [4, 5, 6]

# list1 + list2: creates [1, 2, 3, 4, 5, 6] in memory
combined = list1 + list2

# chain: returns an iterator, avoiding memory allocation for a new list
chained = chain(list1, list2)

2. Lazy Evaluation

   • chain computes elements only when requested (e.g., in a loop or with next()). This is ideal for large datasets or infinite sequences.
   • list1 + list2 creates the result immediately, even if you only need to process part of it.

from itertools import chain
for num in chain(list1, list2):
    print(num)  # Outputs 1, 2, 3, 4, 5, 6 without creating a list

3. Flexibility with Different Iterables

   • chain accepts any iterable objects (lists, tuples, sets, generators, etc.), and there’s no limit to how many you can combine.
   • The + operator works only with two lists (or objects of the same type that support concatenation).

4. Performance with Multiple Lists

   • When combining many lists, repeated use of + (e.g., list1 + list2 + list3 + ...) creates intermediate lists at each step, which is inefficient.
   • chain(list1, list2, list3, ...) merges them in a single pass without extra copies.

If you’re working with max() or min() and need to find a value among elements of multiple lists, chain is preferable for large datasets:

When list1 + list2 Is Better

• If the lists are small and you need a ready-to-use list for further operations (e.g., indexing or repeated use), list1 + list2 is simpler and faster since it doesn’t require converting an iterator to a list.
• chain requires a call to list() or iteration to get the full result, adding slight overhead.

So, which should you choose—chain or list addition?

• Use chain if:
  • You’re working with large datasets or many lists.
  • You need lazy evaluation or memory efficiency.
  • You’re combining different types of iterables (e.g., tuples or sets).
• Use list1 + list2 if:
  • The lists are small, and you need a finished list.
  • Code simplicity and readability matter more.

Practice: Merging Two Lists with Duplicate Removal

A common task is to merge two lists while removing duplicates, creating a new list where each element appears only once. Here are several approaches to solve this.

Approach 1: Using a Loop and Conditions

This method is great for beginners because it’s straightforward and clearly shows the logic. We create a new list and add elements from both lists, checking if they’re already present.

👣 Explanation:

• Create an empty list result.
• Iterate over each element in list1 and add it to result only if it’s not already there.
• Do the same for list2.
• The in operator checks for the presence of an element, and append() adds new elements.

This isn’t the fastest method, especially for large lists, since in is slow for lists.

Approach 2: Using Sets

👣 Explanation:

• set(list1) and set(list2) convert the lists to sets.
• The | operator performs a union of the sets, automatically removing duplicates.
• list() converts the result back to a list.

This is concise and efficient, especially for large lists.

Approach 3: Preserving Order with dict.fromkeys()

👣 Explanation:

• list1 + list2 combines the lists into one (with duplicates).
• dict.fromkeys() creates a dictionary where the keys are the list elements, automatically removing duplicates (dictionary keys are unique).
• list() extracts the keys back into a list.

This is compact and preserves the order of elements.

Which List Merging Method Should You Choose?

Let’s summarize:

Try each method in practice to see which one best fits your needs.

Frequently Asked Questions About Merging Lists

What’s the easiest way to combine two lists in Python?

The easiest way is to use the + operator. For example, list1 + list2 creates a new list with all elements from both lists.

Can I merge lists into a nested list instead of a flat one?

Yes, use .append() to create a list of lists. This keeps the original lists as separate sublists instead of flattening them.