diff --git a/3Sum and 4Sum/index.html b/3Sum and 4Sum/index.html new file mode 100644 index 000000000..3642cca80 --- /dev/null +++ b/3Sum and 4Sum/index.html @@ -0,0 +1,129 @@ + + + 3Sum and 4Sum + + + +

3Sum and 4Sum

+

3Sum and 4Sum are extensions of a popular algorithm question, the 2Sum.

+

3Sum

+
+

Given an array of integers, find all subsets of the array with 3 values where the 3 values sum up to a target number.

+

Note: The solution subsets must not contain duplicate triplets.

+

For example, given the array [-1, 0, 1, 2, -1, -4], and the target 0:
+The solution set is: [[-1, 0, 1], [-1, -1, 2]] // The two -1 values in the array are considered to be distinct

+
+

There are 2 key procedures in solving this algorithm. Sorting the array, and avoiding duplicates.

+

Sorting

+

Sorting your input array allows for powerful assumptions:

+ +

You'll make use of these two rules to create an efficient algorithm.

+

Avoiding Duplicates

+

Since you pre-sort the array, duplicates will be adjacent to each other. You just need to skip over duplicates by comparing adjacent values:

+
extension Collection where Element: Equatable {
+  
+  /// In a sorted collection, replaces the given index with a successor mapping to a unique element.
+  ///
+  /// - Parameter index: A valid index of the collection. `index` must be less than `endIndex`
+  func formUniqueIndex(after index: inout Index) {
+    var prev = index
+    repeat {
+      prev = index
+      formIndex(after: &index)
+    } while index < endIndex && self[prev] == self[index]
+  }
+}
+

A similar implementation is used to get the unique index before a given index:

+
extension BidirectionalCollection where Element: Equatable {
+  
+  /// In a sorted collection, replaces the given index with a predecessor that maps to a unique element.
+  ///
+  /// - Parameter index: A valid index of the collection. `index` must be greater than `startIndex`.
+  func formUniqueIndex(before index: inout Index) {
+    var prev = index
+    repeat {
+      prev = index
+      formIndex(before: &index)
+    } while index > startIndex && self[prev] == self[index]
+  }
+}
+

Assembling the Subsets

+

You'll keep track of 3 indices to represent the 3 numbers. The sum at any given moment is array[l] + array[m] + array[r]:

+
      m ->      <- r
+[-4, -1, -1, 0, 1, 2]
+  l
+

The premise is quite straightforward (given that you're familiar with 2Sum). You'll iterate l through the array. For every iteration, you also apply the 2Sum algorithm to elements after l. You'll check the sum every time you moving the indices to check if you found match. Here's the algorithm:

+
func threeSum<T: BidirectionalCollection>(_ collection: T, target: T.Element) -> [[T.Element]] where T.Element: Numeric & Comparable {
+  let sorted = collection.sorted()
+  var ret: [[T.Element]] = []
+  var l = sorted.startIndex
+  
+  ThreeSum: while l < sorted.endIndex { defer { sorted.formUniqueIndex(after: &l) }
+    var m = sorted.index(after: l)
+    var r = sorted.index(before: sorted.endIndex)
+    
+    TwoSum: while m < r && r < sorted.endIndex { 
+      let sum = sorted[l] + sorted[m] + sorted[r]
+      if sum == target {
+        ret.append([sorted[l], sorted[m], sorted[r]])
+        sorted.formUniqueIndex(after: &m)
+        sorted.formUniqueIndex(before: &r)
+      } else if sum < target {
+        sorted.formUniqueIndex(after: &m)
+      } else {
+        sorted.formUniqueIndex(before: &r)
+      }
+    }
+  }
+  
+  return ret
+}
+

4Sum

+
+

Given an array S of n integers, find all subsets of the array with 4 values where the 4 values sum up to a target number.

+

Note: The solution set must not contain duplicate quadruplets.

+
+

Solution

+

Foursum is a very straightforward extension to the threeSum algorithm. In threeSum, you kept track of 3 indices:

+
      m ->      <- r
+[-4, -1, -1, 0, 1, 2]
+  l
+

For fourSum, you'll keep track of 4:

+
         mr ->  <- r
+[-4, -1, -1, 0, 1, 2]
+  l  ml ->
+

Here's the code for it (notice it is very similar to 3Sum):

+
func fourSum<T: BidirectionalCollection>(_ collection: T, target: T.Element) -> [[T.Element]] where T.Element: Numeric & Comparable {
+  let sorted = collection.sorted()
+  var ret: [[T.Element]] = []
+  
+  var l = sorted.startIndex
+  FourSum: while l < sorted.endIndex { defer { sorted.formUniqueIndex(after: &l) }
+    var ml = sorted.index(after: l)
+    
+    ThreeSum: while ml < sorted.endIndex { defer { sorted.formUniqueIndex(after: &ml) }
+      var mr = sorted.index(after: ml)
+      var r = sorted.index(before: sorted.endIndex)
+      
+      TwoSum: while mr < r && r < sorted.endIndex {
+        let sum = sorted[l] + sorted[ml] + sorted[mr] + sorted[r]
+        if sum == target {
+          ret.append([sorted[l], sorted[ml], sorted[mr], sorted[r]])
+          sorted.formUniqueIndex(after: &mr)
+          sorted.formUniqueIndex(before: &r)
+        } else if sum < target {
+          sorted.formUniqueIndex(after: &mr)
+        } else {
+          sorted.formUniqueIndex(before: &r)
+        }
+      }
+    }
+  }
+  return ret
+}
+

Written for the Swift Algorithm Club by Kai Chen and Kelvin Lau

+ + diff --git a/AVL Tree/index.html b/AVL Tree/index.html new file mode 100644 index 000000000..4a04029e5 --- /dev/null +++ b/AVL Tree/index.html @@ -0,0 +1,12 @@ + + + AVL Tree + + + +
{ + "message": "Problems parsing JSON", + "documentation_url": "https://developer.github.com/v3/markdown/#render-an-arbitrary-markdown-document" +}
+ + diff --git a/Algorithm Design.html b/Algorithm Design.html new file mode 100644 index 000000000..c4edcc819 --- /dev/null +++ b/Algorithm Design.html @@ -0,0 +1,26 @@ + + + Algorithm Design + + + +

Algorithm design techniques

+

What to do when you're faced with a new problem and you need to find an algorithm for it.

+

Is it similar to another problem?

+

If you can frame your problem in terms of another, more general problem, then you might be able to use an existing algorithm. Why reinvent the wheel?

+

One thing I like about The Algorithm Design Manual by Steven Skiena is that it includes a catalog of problems and solutions you can try. (See also his algorithm repository.)

+

It's OK to start with brute force

+

Naive, brute force solutions are often too slow for practical use but they're a good starting point. By writing the brute force solution, you learn to understand what the problem is really all about.

+

Once you have a brute force implementation you can use that to verify that any improvements you come up with are correct.

+

And if you only work with small datasets, then a brute force approach may actually be good enough on its own. Don't fall into the trap of premature optimization!

+

Divide and conquer

+
+

"When you change the way you look at things, the things you look at change."

+Max Planck, Quantum theorist and Nobel Prize Winner

+
+

Divide and conquer is a way of dealing with a large problem by breaking it down into bits and pieces and working your way up towards the solution.

+

Instead of seeing the whole problem as a single, huge and complex task you divide the problem in relatively smaller problems that are easier to understand and deal with.

+

You solve smaller problems and aggregate the solution until you are left with the solution only. At each step the problem at hand shrinks and the solution gets mature until you have the final correct solution.

+

Solving the smaller task and applying the same solution repetitively ( or often times recursively) to other chunks give you the result in less time.

+ + diff --git a/All-Pairs Shortest Paths/index.html b/All-Pairs Shortest Paths/index.html new file mode 100644 index 000000000..14810e728 --- /dev/null +++ b/All-Pairs Shortest Paths/index.html @@ -0,0 +1,59 @@ + + + All-Pairs Shortest Paths + + + +

All-Pairs Shortest Paths

+

The All-Pairs shortest path problem simultaneously computes the shortest path from each node in the graph to each other node, provided a path exists for each pair. In the naive approach, we could simply compute a single-source shortest path from each node to each other node. The number of shortest paths computed is then bound by O(V^2), where V is the number of vertices in the graph. Because SSSP is also bounded by O(V^2), the total running time for the naive approach would be O(V^4).

+

However, by applying a dynamic approach on the adjacency matrix of the graph, a running time of O(V^3) is achievable, using the Floyd-Warshall algorithm. Floyd-Warshall iterates through an adjacency matrix, and for each pair of start(i) and end(j) vertices it considers if the current distance between them is greater than a path taken through another vertex(k) in the graph (if paths i ~> k and k ~> j exist). It moves through an adjacency matrix for every vertex k applying its comparison for each pair (i, j), so for each k a new adjacency matrix D(k) is derived, where each value d(k)[i][j] is defined as:

+

+

where w[i][j] is the weight of the edge connecting vertex i to vertex j in the graph's original adjacency matrix.

+

When the algorithm memoizes each refined adjacency and predecessor matrix, its space complexity is O(V^3), which can be optimised to O(V^2) by only memoizing the latest refinements. Reconstructing paths is a recursive procedure which requires O(V) time and O(V^2) space.

+

Example

+

For the following weighted directed graph

+

+

the adjacency matrix representation w is

+

+

Calculating shortest paths' weights

+

At the beginning of the algorithm, D(0) is the same as w, with the following exceptions to accommodate the comparison function:

+
    +
  1. vertices with no path connecting them have the ø replaced with ∞
    2. +
  2. the diagonal has all 0s
    3. +
+

Here are all the adjacency matrices derived when perform Floyd-Warshall on the above graph:

+

+

+

+

+

with the last being the result, which tells for each pair of starting and ending vertices, the total weight of the shortest path connecting them. During the step where k = 2, the comparison that winds up changing the top right value from 2 to -4 goes like this:

+
k = 2, i = 0, j = 3
+d(k-1)[i][j] => d(1)[0][3] = 2
+d(k-1)[i][k] => d(1)[0][2] = 1
+d(k-1)[j][k] => d(1)[2][3] = -5
+
+

therefore min(2, 2 + -5) => min(2, -4) produces a new weight of -4 for the element at d(2)[0][3], meaning that the shortest known path 1 ~> 4 before this step was from 1 -> 2 -> 4 with a total weight of -2, and afterwards we discovered a shorter path from 1 -> 3 -> 4 with a total weight of -4.

+

Reconstructing shortest paths

+

This algorithm finds only the lengths of the shortest paths between all pairs of nodes; a separate bookkeeping structure must be maintained to track predecessors' indices and reconstruct the shortest paths afterwards. The predecessor matrices at each step for the above graph follow:

+

+

+

+

+

Project Structure

+

The provided xcworkspace allows working in the playground, which imports the APSP framework target from the xcodeproj. Build the framework target and rerun the playground to get started. There is also a test target in the xcodeproj.

+

In the framework:

+ +

TODO

+ +

References

+

Chapter 25 of Introduction to Algorithms, Third Edition by Cormen, Leiserson, Rivest and Stein https://mitpress.mit.edu/books/introduction-algorithms

+

Written for Swift Algorithm Club by Andrew McKnight

+ + diff --git a/Array2D/index.html b/Array2D/index.html new file mode 100644 index 000000000..5f24101b7 --- /dev/null +++ b/Array2D/index.html @@ -0,0 +1,73 @@ + + + Array2D + + + +

Array2D

+

In C and Objective-C, you can write the following line,

+
int cookies[9][7];
+
+

to make a 9x7 grid of cookies. This creates a two-dimensional array of 63 elements. To find the cookie at column 3 and row 6, you can write:

+
myCookie = cookies[3][6];
+
+

This statement is not acceptable in Swift. To create a multi-dimensional array in Swift, you can write:

+
var cookies = [[Int]]()
+for _ in 1...9 {
+  var row = [Int]()
+  for _ in 1...7 {
+    row.append(0)
+  }
+  cookies.append(row)
+}
+

Then, to find a cookie, you can write:

+
let myCookie = cookies[3][6]
+

You can also create the array in a single line of code:

+
var cookies = [[Int]](repeating: [Int](repeating: 0, count: 7), count: 9)
+

This looks complicated, but you can simplify it with a helper function:

+
func dim<T>(_ count: Int, _ value: T) -> [T] {
+  return [T](repeating: value, count: count)
+}
+

Then, you can create the array:

+
var cookies = dim(9, dim(7, 0))
+

Swift infers that the datatype of the array must be Int because you specified 0 as the default value of the array elements. To use a string instead, you can write:

+
var cookies = dim(9, dim(7, "yum"))
+

The dim() function makes it easy to go into even more dimensions:

+
var threeDimensions = dim(2, dim(3, dim(4, 0)))
+

The downside of using multi-dimensional arrays or multiple nested arrays in this way is to lose track of what dimension represents what.

+

Instead, you can create your own type that acts like a 2-D array which is more convenient to use:

+
public struct Array2D<T> {
+  public let columns: Int
+  public let rows: Int
+  fileprivate var array: [T]
+  
+  public init(columns: Int, rows: Int, initialValue: T) {
+    self.columns = columns
+    self.rows = rows
+    array = .init(repeating: initialValue, count: rows*columns)
+  }
+  
+  public subscript(column: Int, row: Int) -> T {
+    get {
+      precondition(column < columns, "Column \(column) Index is out of range. Array<T>(columns: \(columns), rows:\(rows))")
+      precondition(row < rows, "Row \(row) Index is out of range. Array<T>(columns: \(columns), rows:\(rows))")
+      return array[row*columns + column]
+    }
+    set {
+      precondition(column < columns, "Column \(column) Index is out of range. Array<T>(columns: \(columns), rows:\(rows))")
+      precondition(row < rows, "Row \(row) Index is out of range. Array<T>(columns: \(columns), rows:\(rows))")
+      array[row*columns + column] = newValue
+    }
+  }
+}
+

Array2D is a generic type, so it can hold any kind of object, not just numbers.

+

To create an instance of Array2D, you can write:

+
var cookies = Array2D(columns: 9, rows: 7, initialValue: 0)
+

By using the subscript function, you can retrieve an object from the array:

+
let myCookie = cookies[column, row]
+

Or, you can change it:

+
cookies[column, row] = newCookie
+

Internally, Array2D uses a single one-dimensional array to store the data. The index of an object in that array is given by (row x numberOfColumns) + column, but as a user of Array2D, you only need to think in terms of "column" and "row", and the details will be done by Array2D. This is the advantage of wrapping primitive types into a wrapper class or struct.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/B-Tree/index.html b/B-Tree/index.html new file mode 100644 index 000000000..abffd8090 --- /dev/null +++ b/B-Tree/index.html @@ -0,0 +1,162 @@ + + + B-Tree + + + +

B-Tree

+

A B-Tree is a self-balancing search tree, in which nodes can have more than two children.

+

Properties

+

A B-Tree of order n satisfies the following properties:

+ +

A second order B-Tree with keys from 1 to 20 looks like this:

+

A B-Tree with 20 keys.

+

The representation of a B-Tree node in code

+
class BTreeNode<Key: Comparable, Value> {
+  unowned var owner: BTree<Key, Value>
+  
+  fileprivate var keys = [Key]()
+  var children: [BTreeNode]?
+  
+  ...
+}
+

The main parts of a node are two arrays:

+ +

Node.

+

Nodes also have a reference to the tree they belong to.
+This is necessary because nodes have to know the order of the tree.

+

Note: The array containing the children is an Optional, because leaf nodes don't have children.

+

Searching

+
    +
  1. Searching for a key k begins at the root node.
    2. +
  2. We perform a linear search on the keys of the node, until we find a key l that is not less than k,
    +or reach the end of the array.
    3. +
  3. If k == l then we have found the key.
    4. +
  4. If k < l: +
      +
    • If the node we are on is not a leaf, then we go to the left child of l, and perform the steps 3 - 5 again.
      • +
    • If we are on a leaf, then k is not in the tree.
      • +
    +
    5. +
  5. If we have reached the end of the array: +
      +
    • If we are on a non-leaf node, then we go to the last child of the node, and perform the steps 3 - 5 again.
      • +
    • If we are on a leaf, then k is not in the tree.
      • +
    +
    6. +
+

The code

+

value(for:) method searches for the given key and if it's in the tree,
+it returns the value associated with it, else it returns nil.

+

Insertion

+

Keys can only be inserted to leaf nodes.

+
    +
  1. Perform a search for the key k we want to insert.
    2. +
  2. If we haven't found it and we are on a leaf node, we can insert it.
    3. +
+ +

After insertion we should check if the number of keys in the node is in the correct range.
+If there are more keys in the node than order*2, we need to split the node.

+

Splitting a node

+
    +
  1. Move up the middle key of the node we want to split, to its parent (if it has one).
    2. +
  2. If it hasn't got a parent(it is the root), then create a new root and insert to it.
    +Also add the old root as the left child of the new root.
    3. +
  3. Split the node into two by moving the keys (and children, if it's a non-leaf) that were after the middle key
    +to a new node.
    4. +
  4. Add the new node as a right child for the key that we have moved up.
    5. +
+

After splitting a node it is possible that the parent node will also contain too many keys, so we need to split it also.
+In the worst case the splitting goes up to the root (in this case the height of the tree increases).

+

An insertion to a first order tree looks like this:

+

Splitting

+

The code

+

The method insert(_:for:) does the insertion.
+After it has inserted a key, as the recursion goes up every node checks the number of keys in its child.
+if a node has too many keys, its parent calls the split(child:atIndex:) method on it.

+

The root node is checked by the tree itself.
+If the root has too many nodes after the insertion the tree calls the splitRoot() method.

+

Removal

+

Keys can only be removed from leaf nodes.

+
    +
  1. Perform a search for the key k we want to remove.
    2. +
  2. If we have found it: +
      +
    • If we are on a leaf node:
      +We can remove the key.
      • +
    • Else:
      +We overwrite k with its inorder predecessor p, then we remove p from the leaf node.
      • +
    +
    3. +
+

After a key have been removed from a node we should check that the node has enough keys.
+If a node has fewer keys than the order of the tree, then we should move a key to it,
+or merge it with one of its siblings.

+

Moving a key to the child

+

If the problematic node has a nearest sibling that has more keys than the order of the tree,
+we should perform this operation on the tree, else we should merge the node with one of its siblings.

+

Let's say the child we want to fix c1 is at index i in its parent node's children array.

+

If the child c2 at index i-1 has more keys than the order of the tree:

+
    +
  1. We move the key at index i-1 from the parent node to the child c1's keys array at index 0.
    2. +
  2. We move the last key from c2 to the parent's keys array at index i-1.
    3. +
  3. (If c1 is non-leaf) We move the last child from c2 to c1's children array at index 0.
    4. +
+

Else:

+
    +
  1. We move the key at index i from the parent node to the end of child c1's keys array.
    2. +
  2. We move the first key from c2 to the parent's keys array at index i.
    3. +
  3. (If c1 isn't a leaf) We move the first child from c2 to the end of c1's children array.
    4. +
+

Moving Key

+

Merging two nodes

+

Let's say we want to merge the child c1 at index i in its parent's children array.

+

If c1 has a left sibling c2:

+
    +
  1. We move the key from the parent at index i-1 to the end of c2's keys array.
    2. +
  2. We move the keys and the children(if it's a non-leaf) from c1 to the end of c2's keys and children array.
    3. +
  3. We remove the child at index i-1 from the parent node.
    4. +
+

Else if c1 only has a right sibling c2:

+
    +
  1. We move the key from the parent at index i to the beginning of c2's keys array.
    2. +
  2. We move the keys and the children(if it's a non-leaf) from c1 to the beginning of c2's keys and children array.
    3. +
  3. We remove the child at index i from the parent node.
    4. +
+

After merging it is possible that now the parent node contains too few keys,
+so in the worst case merging also can go up to the root, in which case the height of the tree decreases.

+

Merging Nodes

+

The code

+ +

See also

+

Wikipedia
+GeeksforGeeks

+

Written for Swift Algorithm Club by Viktor Szilárd Simkó

+ + diff --git a/Big-O Notation.html b/Big-O Notation.html new file mode 100644 index 000000000..f6d9380e6 --- /dev/null +++ b/Big-O Notation.html @@ -0,0 +1,135 @@ + + + Big-O Notation + + + +

A note on Big-O notation

+

It's useful to know how fast an algorithm is and how much space it needs. This allows you to pick the right algorithm for the job.

+

Big-O notation gives you a rough indication of the running time of an algorithm and the amount of memory it uses. When someone says, "This algorithm has worst-case running time of O(n^2) and uses O(n) space," they mean it's kinda slow but doesn't need lots of extra memory.

+

Figuring out the Big-O of an algorithm is usually done through mathematical analysis. We're skipping the math here, but it's useful to know what the different values mean, so here's a handy table. n refers to the number of data items that you're processing. For example, when sorting an array of 100 items, n = 100.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Big-ONameDescription
O(1)constantThis is the best. The algorithm always takes the same amount of time, regardless of how much data there is. Example: looking up an element of an array by its index.
O(log n)logarithmicPretty great. These kinds of algorithms halve the amount of data with each iteration. If you have 100 items, it takes about 7 steps to find the answer. With 1,000 items, it takes 10 steps. And 1,000,000 items only take 20 steps. This is super fast even for large amounts of data. Example: binary search.
O(n)linearGood performance. If you have 100 items, this does 100 units of work. Doubling the number of items makes the algorithm take exactly twice as long (200 units of work). Example: sequential search.
O(n log n)"linearithmic"Decent performance. This is slightly worse than linear but not too bad. Example: the fastest general-purpose sorting algorithms.
O(n^2)quadraticKinda slow. If you have 100 items, this does 100^2 = 10,000 units of work. Doubling the number of items makes it four times slower (because 2 squared equals 4). Example: algorithms using nested loops, such as insertion sort.
O(n^3)cubicPoor performance. If you have 100 items, this does 100^3 = 1,000,000 units of work. Doubling the input size makes it eight times slower. Example: matrix multiplication.
O(2^n)exponentialVery poor performance. You want to avoid these kinds of algorithms, but sometimes you have no choice. Adding just one bit to the input doubles the running time. Example: traveling salesperson problem.
O(n!)factorialIntolerably slow. It literally takes a million years to do anything.
+

Comparison of Big O computations

+

Below are some examples for each category of performance:

+

O(1)

+

The most common example with O(1) complexity is accessing an array index.

+
let value = array[5]
+

Another example of O(1) is pushing and popping from Stack.

+

O(log n)

+
var j = 1
+while j < n {
+  // do constant time stuff
+  j *= 2
+}
+

Instead of simply incrementing, 'j' is increased by 2 times itself in each run.

+

Binary Search Algorithm is an example of O(log n) complexity.

+

O(n)

+
for i in stride(from: 0, to: n, by: 1) {
+  print(array[i])
+}
+

Array Traversal and Linear Search are examples of O(n) complexity.

+

O(n log n)

+
for i in stride(from: 0, to: n, by: 1) {
+var j = 1
+  while j < n {
+    j *= 2
+    // do constant time stuff
+  }
+}
+

OR

+
for i in stride(from: 0, to: n, by: 1) {
+  func index(after i: Int) -> Int? { // multiplies `i` by 2 until `i` >= `n`
+    return i < n ? i * 2 : nil
+  }
+  for j in sequence(first: 1, next: index(after:)) {
+    // do constant time stuff
+  }
+}
+

Merge Sort and Heap Sort are examples of O(n log n) complexity.

+

O(n^2)

+
for i  in stride(from: 0, to: n, by: 1) {
+  for j in stride(from: 1, to: n, by: 1) {
+    // do constant time stuff
+  }
+}
+

Traversing a simple 2-D array and Bubble Sort are examples of O(n^2) complexity.

+

O(n^3)

+
for i in stride(from: 0, to: n, by: 1) {
+  for j in stride(from: 1, to: n, by: 1) {
+    for k in stride(from: 1, to: n, by: 1) {
+      // do constant time stuff
+    }
+  }
+}
+

O(2^n)

+

Algorithms with running time O(2^N) are often recursive algorithms that solve a problem of size N by recursively solving two smaller problems of size N-1.
+The following example prints all the moves necessary to solve the famous "Towers of Hanoi" problem for N disks.

+
func solveHanoi(n: Int, from: String, to: String, spare: String) {
+  guard n >= 1 else { return }
+  if n > 1 {
+      solveHanoi(n: n - 1, from: from, to: spare, spare: to)
+      solveHanoi(n: n - 1, from: spare, to: to, spare: from)
+  }
+}
+

O(n!)

+

The most trivial example of function that takes O(n!) time is given below.

+
func nFactFunc(n: Int) {
+  for i in stride(from: 0, to: n, by: 1) {
+    nFactFunc(n: n - 1)
+  }
+}
+

Often you don't need math to figure out what the Big-O of an algorithm is but you can simply use your intuition. If your code uses a single loop that looks at all n elements of your input, the algorithm is O(n). If the code has two nested loops, it is O(n^2). Three nested loops gives O(n^3), and so on.

+

Note that Big-O notation is an estimate and is only really useful for large values of n. For example, the worst-case running time for the insertion sort algorithm is O(n^2). In theory that is worse than the running time for merge sort, which is O(n log n). But for small amounts of data, insertion sort is actually faster, especially if the array is partially sorted already!

+

If you find this confusing, don't let this Big-O stuff bother you too much. It's mostly useful when comparing two algorithms to figure out which one is better. But in the end you still want to test in practice which one really is the best. And if the amount of data is relatively small, then even a slow algorithm will be fast enough for practical use.

+ + diff --git a/Binary Search Tree/index.html b/Binary Search Tree/index.html new file mode 100644 index 000000000..cf443e1b9 --- /dev/null +++ b/Binary Search Tree/index.html @@ -0,0 +1,526 @@ + + + Binary Search Tree + + + +

Binary Search Tree (BST)

+
+

This topic has been tutorialized here

+
+

A binary search tree is a special kind of binary tree (a tree in which each node has at most two children) that performs insertions and deletions such that the tree is always sorted.

+

For more information about a tree, read this first.

+

"Always sorted" property

+

Here is an example of a valid binary search tree:

+

A binary search tree

+

Notice how each left child is smaller than its parent node, and each right child is greater than its parent node. This is the key feature of a binary search tree.

+

For example, 2 is smaller than 7, so it goes on the left; 5 is greater than 2, so it goes on the right.

+

Inserting new nodes

+

When performing an insertion, we first compare the new value to the root node. If the new value is smaller, we take the left branch; if greater, we take the right branch. We work our way down the tree this way until we find an empty spot where we can insert the new value.

+

Suppose we want to insert the new value 9:

+ +

Here is the tree after inserting the new value 9:

+

After adding 9

+

There is only one possible place where the new element can be inserted in the tree. Finding this place is usually quick. It takes O(h) time, where h is the height of the tree.

+
+

Note: The height of a node is the number of steps it takes to go from that node to its lowest leaf. The height of the entire tree is the distance from the root to the lowest leaf. Many of the operations on a binary search tree are expressed in terms of the tree's height.

+
+

By following this simple rule -- smaller values on the left, larger values on the right -- we keep the tree sorted, so whenever we query it, we can check if a value is in the tree.

+

Searching the tree

+

To find a value in the tree, we perform the same steps as with insertion:

+ +

Like most tree operations, this is performed recursively until either we find what we are looking for or run out of nodes to look at.

+

Here is an example for searching the value 5:

+

Searching the tree

+

Searching is fast using the structure of the tree. It runs in O(h) time. If you have a well-balanced tree with a million nodes, it only takes about 20 steps to find anything in this tree. (The idea is very similar to binary search in an array.)

+

Traversing the tree

+

Sometimes you need to look at all nodes rather than only one.

+

There are three ways to traverse a binary tree:

+
    +
  1. In-order (or depth-first): first look at the left child of a node then at the node itself and finally at its right child.
    2. +
  2. Pre-order: first look at a node then its left and right children.
    3. +
  3. Post-order: first look at the left and right children and process the node itself last.
    4. +
+

Traversing the tree also happens recursively.

+

If you traverse a binary search tree in-order, it looks at all the nodes as if they were sorted from low to high. For the example tree, it would print 1, 2, 5, 7, 9, 10:

+

Traversing the tree

+

Deleting nodes

+

Removing nodes is easy. After removing a node, we replace the node with either its biggest child on the left or its smallest child on the right. That way the tree is still sorted after the removal. In the following example, 10 is removed and replaced with either 9 (Figure 2) or 11 (Figure 3).

+

Deleting a node with two children

+

Note the replacement needs to happen when the node has at least one child. If it has no child, you just disconnect it from its parent:

+

Deleting a leaf node

+

The code (solution 1)

+

So much for the theory. Let's see how we can implement a binary search tree in Swift. There are different approaches you can take. First, I will show you how to make a class-based version, but we will also look at how to make one using enums.

+

Here is an example for a BinarySearchTree class:

+
public class BinarySearchTree<T: Comparable> {
+  private(set) public var value: T
+  private(set) public var parent: BinarySearchTree?
+  private(set) public var left: BinarySearchTree?
+  private(set) public var right: BinarySearchTree?
+
+  public init(value: T) {
+    self.value = value
+  }
+
+  public var isRoot: Bool {
+    return parent == nil
+  }
+
+  public var isLeaf: Bool {
+    return left == nil && right == nil
+  }
+
+  public var isLeftChild: Bool {
+    return parent?.left === self
+  }
+
+  public var isRightChild: Bool {
+    return parent?.right === self
+  }
+
+  public var hasLeftChild: Bool {
+    return left != nil
+  }
+
+  public var hasRightChild: Bool {
+    return right != nil
+  }
+
+  public var hasAnyChild: Bool {
+    return hasLeftChild || hasRightChild
+  }
+
+  public var hasBothChildren: Bool {
+    return hasLeftChild && hasRightChild
+  }
+
+  public var count: Int {
+    return (left?.count ?? 0) + 1 + (right?.count ?? 0)
+  }
+}
+

This class describes just a single node not the entire tree. It is a generic type, so the node can store any kind of data. It also has references to its left and right child nodes and a parent node.

+

Here is how you can use it:

+
let tree = BinarySearchTree<Int>(value: 7)
+

The count property determines how many nodes are in the subtree described by this node. This does not just count the node's immediate children but also their children and their children's children, and so on. If this particular object is the root node, then it counts how many nodes are in the entire tree. Initially, count = 0.

+
+

Note: Because left, right, and parent are optional, we can make good use of Swift's optional chaining (?) and nil-coalescing operators (??). You could also write this sort of thing with if let, but that is less concise.

+
+

Inserting nodes

+

A tree node by itself is useless, so here is how you would add new nodes to the tree:

+
  public func insert(value: T) {
+    if value < self.value {
+      if let left = left {
+        left.insert(value: value)
+      } else {
+        left = BinarySearchTree(value: value)
+        left?.parent = self
+      }
+    } else {
+      if let right = right {
+        right.insert(value: value)
+      } else {
+        right = BinarySearchTree(value: value)
+        right?.parent = self
+      }
+    }
+  }
+

Like so many other tree operations, insertion is easiest to implement with recursion. We compare the new value to the values of the existing nodes and decide whether to add it to the left branch or the right branch.

+

If there is no more left or right child to look at, we create a BinarySearchTree object for the new node and connect it to the tree by setting its parent property.

+
+

Note: Because the whole point of a binary search tree is to have smaller nodes on the left and larger ones on the right, you should always insert elements at the root to make sure this remains a valid binary tree!

+
+

To build the complete tree from the example you can do:

+
let tree = BinarySearchTree<Int>(value: 7)
+tree.insert(2)
+tree.insert(5)
+tree.insert(10)
+tree.insert(9)
+tree.insert(1)
+
+

Note: For reasons that will become clear later, you should insert the numbers in a random order. If you insert them in a sorted order, the tree will not have the right shape.

+
+

For convenience, let's add an init method that calls insert() for all the elements in an array:

+
  public convenience init(array: [T]) {
+    precondition(array.count > 0)
+    self.init(value: array.first!)
+    for v in array.dropFirst() {
+      insert(value: v)
+    }
+  }
+

Now you can simply do this:

+
let tree = BinarySearchTree<Int>(array: [7, 2, 5, 10, 9, 1])
+

The first value in the array becomes the root of the tree.

+

Debug output

+

When working with complicated data structures, it is useful to have human-readable debug output.

+
extension BinarySearchTree: CustomStringConvertible {
+  public var description: String {
+    var s = ""
+    if let left = left {
+      s += "(\(left.description)) <- "
+    }
+    s += "\(value)"
+    if let right = right {
+      s += " -> (\(right.description))"
+    }
+    return s
+  }
+}
+

When you do a print(tree), you should get something like this:

+
((1) <- 2 -> (5)) <- 7 -> ((9) <- 10)
+
+

The root node is in the middle. With some imagination, you should see that this indeed corresponds to the following tree:

+

The tree

+

You may be wondering what happens when you insert duplicate items? We always insert those in the right branch. Try it out!

+

Searching

+

What do we do now that we have some values in our tree? Search for them, of course! To find items quickly is the main purpose of a binary search tree. :-)

+

Here is the implementation of search():

+
  public func search(value: T) -> BinarySearchTree? {
+    if value < self.value {
+      return left?.search(value)
+    } else if value > self.value {
+      return right?.search(value)
+    } else {
+      return self  // found it!
+    }
+  }
+

I hope the logic is clear: this starts at the current node (usually the root) and compares the values. If the search value is less than the node's value, we continue searching in the left branch; if the search value is greater, we dive into the right branch.

+

If there are no more nodes to look at -- when left or right is nil -- then we return nil to indicate the search value is not in the tree.

+
+

Note: In Swift, that is conveniently done with optional chaining; when you write left?.search(value) it automatically returns nil if left is nil. There is no need to explicitly check for this with an if statement.

+
+

Searching is a recursive process, but you can also implement it with a simple loop instead:

+
  public func search(_ value: T) -> BinarySearchTree? {
+    var node: BinarySearchTree? = self
+    while let n = node {
+      if value < n.value {
+        node = n.left
+      } else if value > n.value {
+        node = n.right
+      } else {
+        return node
+      }
+    }
+    return nil
+  }
+

Verify that you understand these two implementations are equivalent. Personally, I prefer to use iterative code over recursive code, but your opinion may differ. ;-)

+

Here is how to test searching:

+
tree.search(5)
+tree.search(2)
+tree.search(7)
+tree.search(6)   // nil
+

The first three lines return the corresponding BinaryTreeNode object. The last line returns nil because there is no node with value 6.

+
+

Note: If there are duplicate items in the tree, search() returns the "highest" node. That makes sense, because we start searching from the root downwards.

+
+

Traversal

+

Remember there are 3 different ways to look at all nodes in the tree? Here they are:

+
  public func traverseInOrder(process: (T) -> Void) {
+    left?.traverseInOrder(process: process)
+    process(value)
+    right?.traverseInOrder(process: process)
+  }
+
+  public func traversePreOrder(process: (T) -> Void) {
+    process(value)
+    left?.traversePreOrder(process: process)
+    right?.traversePreOrder(process: process)
+  }
+
+  public func traversePostOrder(process: (T) -> Void) {
+    left?.traversePostOrder(process: process)
+    right?.traversePostOrder(process: process)
+    process(value)
+  }
+

They all work the same but in different orders. Notice that all the work is done recursively. The Swift's optional chaining makes it clear that the calls to traverseInOrder() etc are ignored when there is no left or right child.

+

To print out all the values of the tree sorted from low to high you can write:

+
tree.traverseInOrder { value in print(value) }
+

This prints the following:

+
1
+2
+5
+7
+9
+10
+
+

You can also add things like map() and filter() to the tree. For example, here is an implementation of map:

+
  public func map(formula: (T) -> T) -> [T] {
+    var a = [T]()
+    if let left = left { a += left.map(formula: formula) }
+    a.append(formula(value))
+    if let right = right { a += right.map(formula: formula) }
+    return a
+  }
+

This calls the formula closure on each node in the tree and collects the results in an array. map() works by traversing the tree in-order.

+

An extremely simple example of how to use map():

+
  public func toArray() -> [T] {
+    return map { $0 }
+  }
+

This turns the contents of the tree back into a sorted array. Try it out in the playground:

+
tree.toArray()   // [1, 2, 5, 7, 9, 10]
+

As an exercise, see if you can implement filter and reduce.

+

Deleting nodes

+

We can make the code more readable by defining some helper functions.

+
  private func reconnectParentTo(node: BinarySearchTree?) {
+    if let parent = parent {
+      if isLeftChild {
+        parent.left = node
+      } else {
+        parent.right = node
+      }
+    }
+    node?.parent = parent
+  }
+

Making changes to the tree involves changing a bunch of parent and left and right pointers. This function helps with this implementation. It takes the parent of the current node -- that is self -- and connects it to another node which will be one of the children of self.

+

We also need a function that returns the minimum and maximum of a node:

+
  public func minimum() -> BinarySearchTree {
+    var node = self
+    while let next = node.left {
+      node = next
+    }
+    return node
+  }
+
+  public func maximum() -> BinarySearchTree {
+    var node = self
+    while let next = node.right {
+      node = next
+    }
+    return node
+  }
+
+

The rest of the code is self-explanatory:

+
  @discardableResult public func remove() -> BinarySearchTree? {
+    let replacement: BinarySearchTree?
+
+    // Replacement for current node can be either biggest one on the left or
+    // smallest one on the right, whichever is not nil
+    if let right = right {
+      replacement = right.minimum()
+    } else if let left = left {
+      replacement = left.maximum()
+    } else {
+      replacement = nil
+    }
+
+    replacement?.remove()
+
+    // Place the replacement on current node's position
+    replacement?.right = right
+    replacement?.left = left
+    right?.parent = replacement
+    left?.parent = replacement
+    reconnectParentTo(node:replacement)
+
+    // The current node is no longer part of the tree, so clean it up.
+    parent = nil
+    left = nil
+    right = nil
+
+    return replacement
+  }
+

Depth and height

+

Recall that the height of a node is the distance to its lowest leaf. We can calculate that with the following function:

+
  public func height() -> Int {
+    if isLeaf {
+      return 0
+    } else {
+      return 1 + max(left?.height() ?? 0, right?.height() ?? 0)
+    }
+  }
+

We look at the heights of the left and right branches and take the highest one. Again, this is a recursive procedure. Since this looks at all children of this node, performance is O(n).

+
+

Note: Swift's null-coalescing operator is used as shorthand to deal with left or right pointers that are nil. You could write this with if let, but this is more concise.

+
+

Try it out:

+
tree.height()  // 2
+

You can also calculate the depth of a node, which is the distance to the root. Here is the code:

+
  public func depth() -> Int {
+    var node = self
+    var edges = 0
+    while let parent = node.parent {
+      node = parent
+      edges += 1
+    }
+    return edges
+  }
+

It steps upwards through the tree, following the parent pointers until we reach the root node (whose parent is nil). This takes O(h) time. Here is an example:

+
if let node9 = tree.search(9) {
+  node9.depth()   // returns 2
+}
+

Predecessor and successor

+

The binary search tree is always "sorted" but that does not mean that consecutive numbers are actually next to each other in the tree.

+

Example

+

Note that you cannot find the number that comes before 7 by just looking at its left child node. The left child is 2, not 5. Likewise for the number that comes after 7.

+

The predecessor() function returns the node whose value precedes the current value in sorted order:

+
  public func predecessor() -> BinarySearchTree<T>? {
+    if let left = left {
+      return left.maximum()
+    } else {
+      var node = self
+      while let parent = node.parent {
+        if parent.value < value { return parent }
+        node = parent
+      }
+      return nil
+    }
+  }
+

It is easy if we have a left subtree. In that case, the immediate predecessor is the maximum value in that subtree. You can verify in the above picture that 5 is indeed the maximum value in 7's left branch.

+

If there is no left subtree, then we have to look at our parent nodes until we find a smaller value. If we want to know what the predecessor is of node 9, we keep going up until we find the first parent with a smaller value, which is 7.

+

The code for successor() works the same way but mirrored:

+
  public func successor() -> BinarySearchTree<T>? {
+    if let right = right {
+      return right.minimum()
+    } else {
+      var node = self
+      while let parent = node.parent {
+        if parent.value > value { return parent }
+        node = parent
+      }
+      return nil
+    }
+  }
+

Both these methods run in O(h) time.

+
+

Note: There is a variation called a "threaded" binary tree where "unused" left and right pointers are repurposed to make direct links between predecessor and successor nodes. Very clever!

+
+

Is the search tree valid?

+

If you were intent on sabotage you could turn the binary search tree into an invalid tree by calling insert() on a node that is not the root. Here is an example:

+
if let node1 = tree.search(1) {
+  node1.insert(100)
+}
+

The value of the root node is 7, so a node with value 100must be in the tree's right branch. However, you are not inserting at the root but at a leaf node in the tree's left branch. So the new 100 node is in the wrong place in the tree!

+

As a result, doing tree.search(100) gives nil.

+

You can check whether a tree is a valid binary search tree with the following method:

+
  public func isBST(minValue minValue: T, maxValue: T) -> Bool {
+    if value < minValue || value > maxValue { return false }
+    let leftBST = left?.isBST(minValue: minValue, maxValue: value) ?? true
+    let rightBST = right?.isBST(minValue: value, maxValue: maxValue) ?? true
+    return leftBST && rightBST
+  }
+

This verifies the left branch contains values that are less than the current node's value, and that the right branch only contains values that are larger.

+

Call it as follows:

+
if let node1 = tree.search(1) {
+  tree.isBST(minValue: Int.min, maxValue: Int.max)  // true
+  node1.insert(100)                                 // EVIL!!!
+  tree.search(100)                                  // nil
+  tree.isBST(minValue: Int.min, maxValue: Int.max)  // false
+}
+

The code (solution 2)

+

We have implemented the binary tree node as a class, but you can also use an enum.

+

The difference is reference semantics versus value semantics. Making a change to the class-based tree will update that same instance in memory, but the enum-based tree is immutable -- any insertions or deletions will give you an entirely new copy of the tree. Which one is best, totally depends on what you want to use it for.

+

Here is how you can make a binary search tree using an enum:

+
public enum BinarySearchTree<T: Comparable> {
+  case Empty
+  case Leaf(T)
+  indirect case Node(BinarySearchTree, T, BinarySearchTree)
+}
+

The enum has three cases:

+ +
+

Note: The nodes in this binary tree do not have a reference to their parent node. It is not a major impediment, but it will make certain operations more cumbersome to implement.

+
+

This implementation is recursive, and each case of the enum will be treated differently. For example, this is how you can calculate the number of nodes in the tree and the height of the tree:

+
  public var count: Int {
+    switch self {
+    case .Empty: return 0
+    case .Leaf: return 1
+    case let .Node(left, _, right): return left.count + 1 + right.count
+    }
+  }
+
+  public var height: Int {
+    switch self {
+    case .Empty: return -1
+    case .Leaf: return 0
+    case let .Node(left, _, right): return 1 + max(left.height, right.height)
+    }
+  }
+

Inserting new nodes looks like this:

+
  public func insert(newValue: T) -> BinarySearchTree {
+    switch self {
+    case .Empty:
+      return .Leaf(newValue)
+
+    case .Leaf(let value):
+      if newValue < value {
+        return .Node(.Leaf(newValue), value, .Empty)
+      } else {
+        return .Node(.Empty, value, .Leaf(newValue))
+      }
+
+    case .Node(let left, let value, let right):
+      if newValue < value {
+        return .Node(left.insert(newValue), value, right)
+      } else {
+        return .Node(left, value, right.insert(newValue))
+      }
+    }
+  }
+

Try it out in a playground:

+
var tree = BinarySearchTree.Leaf(7)
+tree = tree.insert(2)
+tree = tree.insert(5)
+tree = tree.insert(10)
+tree = tree.insert(9)
+tree = tree.insert(1)
+

Notice that for each insertion, you get back a new tree object, so you need to assign the result back to the tree variable.

+

Here is the all-important search function:

+
  public func search(x: T) -> BinarySearchTree? {
+    switch self {
+    case .Empty:
+      return nil
+    case .Leaf(let y):
+      return (x == y) ? self : nil
+    case let .Node(left, y, right):
+      if x < y {
+        return left.search(x)
+      } else if y < x {
+        return right.search(x)
+      } else {
+        return self
+      }
+    }
+  }
+

Most of these functions have the same structure.

+

Try it out in a playground:

+
tree.search(10)
+tree.search(1)
+tree.search(11)   // nil
+

To print the tree for debug purposes, you can use this method:

+
extension BinarySearchTree: CustomDebugStringConvertible {
+  public var debugDescription: String {
+    switch self {
+    case .Empty: return "."
+    case .Leaf(let value): return "\(value)"
+    case .Node(let left, let value, let right):
+      return "(\(left.debugDescription) <- \(value) -> \(right.debugDescription))"
+    }
+  }
+}
+

When you do print(tree), it will look like this:

+
((1 <- 2 -> 5) <- 7 -> (9 <- 10 -> .))
+
+

The root node is in the middle, and a dot means there is no child at that position.

+

When the tree becomes unbalanced...

+

A binary search tree is balanced when its left and right subtrees contain the same number of nodes. In that case, the height of the tree is log(n), where n is the number of nodes. That is the ideal situation.

+

If one branch is significantly longer than the other, searching becomes very slow. We end up checking more values than we need. In the worst case, the height of the tree can become n. Such a tree acts like a linked list than a binary search tree, with performance degrading to O(n). Not good!

+

One way to make the binary search tree balanced is to insert the nodes in a totally random order. On average that should balance out the tree well, but it not guaranteed, nor is it always practical.

+

The other solution is to use a self-balancing binary tree. This type of data structure adjusts the tree to keep it balanced after you insert or delete nodes. To see examples, check AVL tree and red-black tree.

+

See also

+

Binary Search Tree on Wikipedia

+

Written for the Swift Algorithm Club by Nicolas Ameghino and Matthijs Hollemans

+ + diff --git a/Binary Search/index.html b/Binary Search/index.html new file mode 100644 index 000000000..ea0af0903 --- /dev/null +++ b/Binary Search/index.html @@ -0,0 +1,153 @@ + + + Binary Search + + + +

Binary Search

+

Goal: Quickly find an element in an array.

+

Let's say you have an array of numbers and you want to determine whether a specific number is in that array, and if so, at which index.

+

In most cases, Swift's Collection.index(of:) function is good enough for that:

+
let numbers = [11, 59, 3, 2, 53, 17, 31, 7, 19, 67, 47, 13, 37, 61, 29, 43, 5, 41, 23]
+
+numbers.index(of: 43)  // returns 15
+

The built-in Collection.index(of:) function performs a linear search. In code that looks something like this:

+
func linearSearch<T: Equatable>(_ a: [T], _ key: T) -> Int? {
+    for i in 0 ..< a.count {
+        if a[i] == key {
+            return i
+        }
+    }
+    return nil
+}
+

And you'd use it like this:

+
linearSearch(numbers, 43)  // returns 15
+

So what's the problem? linearSearch() loops through the entire array from the beginning, until it finds the element you're looking for. In the worst case, the value isn't even in the array and all that work is done for nothing.

+

On average, the linear search algorithm needs to look at half the values in the array. If your array is large enough, this starts to become very slow!

+

Divide and conquer

+

The classic way to speed this up is to use a binary search. The trick is to keep splitting the array in half until the value is found.

+

For an array of size n, the performance is not O(n) as with linear search but only O(log n). To put that in perspective, binary search on an array with 1,000,000 elements only takes about 20 steps to find what you're looking for, because log_2(1,000,000) = 19.9. And for an array with a billion elements it only takes 30 steps. (Then again, when was the last time you used an array with a billion items?)

+

Sounds great, but there is a downside to using binary search: the array must be sorted. In practice, this usually isn't a problem.

+

Here's how binary search works:

+ +

Now you know why it's called a "binary" search: in every step it splits the array into two halves. This process of divide-and-conquer is what allows it to quickly narrow down where the search key must be.

+

The code

+

Here is a recursive implementation of binary search in Swift:

+
func binarySearch<T: Comparable>(_ a: [T], key: T, range: Range<Int>) -> Int? {
+    if range.lowerBound >= range.upperBound {
+        // If we get here, then the search key is not present in the array.
+        return nil
+
+    } else {
+        // Calculate where to split the array.
+        let midIndex = range.lowerBound + (range.upperBound - range.lowerBound) / 2
+
+        // Is the search key in the left half?
+        if a[midIndex] > key {
+            return binarySearch(a, key: key, range: range.lowerBound ..< midIndex)
+
+        // Is the search key in the right half?
+        } else if a[midIndex] < key {
+            return binarySearch(a, key: key, range: midIndex + 1 ..< range.upperBound)
+
+        // If we get here, then we've found the search key!
+        } else {
+            return midIndex
+        }
+    }
+}
+

To try this out, copy the code to a playground and do:

+
let numbers = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67]
+
+binarySearch(numbers, key: 43, range: 0 ..< numbers.count)  // gives 13
+

Note that the numbers array is sorted. The binary search algorithm does not work otherwise!

+

I said that binary search works by splitting the array in half, but we don't actually create two new arrays. Instead, we keep track of these splits using a Swift Range object. Initially, this range covers the entire array, 0 ..< numbers.count. As we split the array, the range becomes smaller and smaller.

+
+

Note: One thing to be aware of is that range.upperBound always points one beyond the last element. In the example, the range is 0..<19 because there are 19 numbers in the array, and so range.lowerBound = 0 and range.upperBound = 19. But in our array the last element is at index 18, not 19, since we start counting from 0. Just keep this in mind when working with ranges: the upperBound is always one more than the index of the last element.

+
+

Stepping through the example

+

It might be useful to look at how the algorithm works in detail.

+

The array from the above example consists of 19 numbers and looks like this when sorted:

+
[ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67 ]
+
+

We're trying to determine if the number 43 is in this array.

+

To split the array in half, we need to know the index of the object in the middle. That's determined by this line:

+
let midIndex = range.lowerBound + (range.upperBound - range.lowerBound) / 2
+

Initially, the range has lowerBound = 0 and upperBound = 19. Filling in these values, we find that midIndex is 0 + (19 - 0)/2 = 19/2 = 9. It's actually 9.5 but because we're using integers, the answer is rounded down.

+

In the next figure, the * shows the middle item. As you can see, the number of items on each side is the same, so we're split right down the middle.

+
[ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67 ]
+                                  *
+
+

Now binary search will determine which half to use. The relevant section from the code is:

+
if a[midIndex] > key {
+    // use left half
+} else if a[midIndex] < key {
+    // use right half
+} else {
+    return midIndex
+}
+

In this case, a[midIndex] = 29. That's less than the search key, so we can safely conclude that the search key will never be in the left half of the array. After all, the left half only contains numbers smaller than 29. Hence, the search key must be in the right half somewhere (or not in the array at all).

+

Now we can simply repeat the binary search, but on the array interval from midIndex + 1 to range.upperBound:

+
[ x, x, x, x, x, x, x, x, x, x | 31, 37, 41, 43, 47, 53, 59, 61, 67 ]
+
+

Since we no longer need to concern ourselves with the left half of the array, I've marked that with x's. From now on we'll only look at the right half, which starts at array index 10.

+

We calculate the index of the new middle element: midIndex = 10 + (19 - 10)/2 = 14, and split the array down the middle again.

+
[ x, x, x, x, x, x, x, x, x, x | 31, 37, 41, 43, 47, 53, 59, 61, 67 ]
+                                                 *
+
+

As you can see, a[14] is indeed the middle element of the array's right half.

+

Is the search key greater or smaller than a[14]? It's smaller because 43 < 47. This time we're taking the left half and ignore the larger numbers on the right:

+
[ x, x, x, x, x, x, x, x, x, x | 31, 37, 41, 43 | x, x, x, x, x ]
+
+

The new midIndex is here:

+
[ x, x, x, x, x, x, x, x, x, x | 31, 37, 41, 43 | x, x, x, x, x ]
+                                     *
+
+

The search key is greater than 37, so continue with the right side:

+
[ x, x, x, x, x, x, x, x, x, x | x, x | 41, 43 | x, x, x, x, x ]
+                                        *
+
+

Again, the search key is greater, so split once more and take the right side:

+
[ x, x, x, x, x, x, x, x, x, x | x, x | x | 43 | x, x, x, x, x ]
+                                            *
+
+

And now we're done. The search key equals the array element we're looking at, so we've finally found what we were searching for: number 43 is at array index 13. w00t!

+

It may have seemed like a lot of work, but in reality it only took four steps to find the search key in the array, which sounds about right because log_2(19) = 4.23. With a linear search, it would have taken 14 steps.

+

What would happen if we were to search for 42 instead of 43? In that case, we can't split up the array any further. The range.upperBound becomes smaller than range.lowerBound. That tells the algorithm the search key is not in the array and it returns nil.

+
+

Note: Many implementations of binary search calculate midIndex = (lowerBound + upperBound) / 2. This contains a subtle bug that only appears with very large arrays, because lowerBound + upperBound may overflow the maximum number an integer can hold. This situation is unlikely to happen on a 64-bit CPU, but it definitely can on 32-bit machines.

+
+

Iterative vs recursive

+

Binary search is recursive in nature because you apply the same logic over and over again to smaller and smaller subarrays. However, that does not mean you must implement binarySearch() as a recursive function. It's often more efficient to convert a recursive algorithm into an iterative version, using a simple loop instead of lots of recursive function calls.

+

Here is an iterative implementation of binary search in Swift:

+
func binarySearch<T: Comparable>(_ a: [T], key: T) -> Int? {
+    var lowerBound = 0
+    var upperBound = a.count
+    while lowerBound < upperBound {
+        let midIndex = lowerBound + (upperBound - lowerBound) / 2
+        if a[midIndex] == key {
+            return midIndex
+        } else if a[midIndex] < key {
+            lowerBound = midIndex + 1
+        } else {
+            upperBound = midIndex
+        }
+    }
+    return nil
+}
+

As you can see, the code is very similar to the recursive version. The main difference is in the use of the while loop.

+

Use it like this:

+
let numbers = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67]
+
+binarySearch(numbers, key: 43)  // gives 13
+

The end

+

Is it a problem that the array must be sorted first? It depends. Keep in mind that sorting takes time -- the combination of binary search plus sorting may be slower than doing a simple linear search. Binary search shines in situations where you sort just once and then do many searches.

+

See also Wikipedia.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Binary Tree/index.html b/Binary Tree/index.html new file mode 100644 index 000000000..684a1fef9 --- /dev/null +++ b/Binary Tree/index.html @@ -0,0 +1,142 @@ + + + Binary Tree + + + +

Binary Tree

+

A binary tree is a tree where each node has 0, 1, or 2 children. This is a binary tree:

+

A binary tree

+

The child nodes are usually called the left child and the right child. If a node doesn't have any children, it's called a leaf node. The root is the node at the very top of the tree (programmers like their trees upside down).

+

Often nodes will have a link back to their parent but this is not strictly necessary.

+

Binary trees are often used as binary search trees. In that case, the nodes must be in a specific order (smaller values on the left, larger values on the right). But this is not a requirement for all binary trees.

+

For example, here is a binary tree that represents a sequence of arithmetical operations, (5 * (a - 10)) + (-4 * (3 / b)):

+

A binary tree

+

The code

+

Here's how you could implement a general-purpose binary tree in Swift:

+
public indirect enum BinaryTree<T> {
+  case node(BinaryTree<T>, T, BinaryTree<T>)
+  case empty
+}
+

As an example of how to use this, let's build that tree of arithmetic operations:

+
// leaf nodes
+let node5 = BinaryTree.node(.empty, "5", .empty)
+let nodeA = BinaryTree.node(.empty, "a", .empty)
+let node10 = BinaryTree.node(.empty, "10", .empty)
+let node4 = BinaryTree.node(.empty, "4", .empty)
+let node3 = BinaryTree.node(.empty, "3", .empty)
+let nodeB = BinaryTree.node(.empty, "b", .empty)
+
+// intermediate nodes on the left
+let Aminus10 = BinaryTree.node(nodeA, "-", node10)
+let timesLeft = BinaryTree.node(node5, "*", Aminus10)
+
+// intermediate nodes on the right
+let minus4 = BinaryTree.node(.empty, "-", node4)
+let divide3andB = BinaryTree.node(node3, "/", nodeB)
+let timesRight = BinaryTree.node(minus4, "*", divide3andB)
+
+// root node
+let tree = BinaryTree.node(timesLeft, "+", timesRight)
+

You need to build up the tree in reverse, starting with the leaf nodes and working your way up to the top.

+

It will be useful to add a description method so you can print the tree:

+
extension BinaryTree: CustomStringConvertible {
+  public var description: String {
+    switch self {
+    case let .node(left, value, right):
+      return "value: \(value), left = [\(left.description)], right = [\(right.description)]"
+    case .empty:
+      return ""
+    }
+  }
+}
+

If you print(tree) you should see something like this:

+
value: +, left = [value: *, left = [value: 5, left = [], right = []], right = [value: -, left = [value: a, left = [], right = []], right = [value: 10, left = [], right = []]]], right = [value: *, left = [value: -, left = [], right = [value: 4, left = [], right = []]], right = [value: /, left = [value: 3, left = [], right = []], right = [value: b, left = [], right = []]]]
+
+

With a bit of imagination, you can see the tree structure. ;-) It helps if you indent it:

+
value: +, 
+	left = [value: *, 
+		left = [value: 5, left = [], right = []], 
+		right = [value: -, 
+			left = [value: a, left = [], right = []], 
+			right = [value: 10, left = [], right = []]]], 
+	right = [value: *, 
+		left = [value: -, 
+			left = [], 
+			right = [value: 4, left = [], right = []]], 
+		right = [value: /, 
+			left = [value: 3, left = [], right = []], 
+			right = [value: b, left = [], right = []]]]
+
+

Another useful method is counting the number of nodes in the tree:

+
  public var count: Int {
+    switch self {
+    case let .node(left, _, right):
+      return left.count + 1 + right.count
+    case .empty:
+      return 0
+    }
+  }
+

On the tree from the example, tree.count should be 12.

+

Something you often need to do with trees is traverse them, i.e. look at all the nodes in some order. There are three ways to traverse a binary tree:

+
    +
  1. In-order (or depth-first): first look at the left child of a node, then at the node itself, and finally at its right child.
    2. +
  2. Pre-order: first look at a node, then at its left and right children.
    3. +
  3. Post-order: first look at the left and right children and process the node itself last.
    4. +
+

Here is how you'd implement that:

+
  public func traverseInOrder(process: (T) -> Void) {
+    if case let .node(left, value, right) = self {
+      left.traverseInOrder(process: process)
+      process(value)
+      right.traverseInOrder(process: process)
+    }
+  }
+  
+  public func traversePreOrder(process: (T) -> Void) {
+    if case let .node(left, value, right) = self {
+      process(value)
+      left.traversePreOrder(process: process)
+      right.traversePreOrder(process: process)
+    }
+  }
+  
+  public func traversePostOrder(process: (T) -> Void) {
+    if case let .node(left, value, right) = self {
+      left.traversePostOrder(process: process)
+      right.traversePostOrder(process: process)
+      process(value)
+    }
+  }
+

As is common when working with tree structures, these functions call themselves recursively.

+

For example, if you traverse the tree of arithmetic operations in post-order, you'll see the values in this order:

+
5
+a
+10
+-
+*
+4
+-
+3
+b
+/
+*
++
+
+

The leaves appear first. The root node appears last.

+

You can use a stack machine to evaluate these expressions, something like the following pseudocode:

+
tree.traversePostOrder { s in 
+  switch s {
+  case this is a numeric literal, such as 5:
+    push it onto the stack
+  case this is a variable name, such as a:
+    look up the value of a and push it onto the stack
+  case this is an operator, such as *:
+    pop the two top-most items off the stack, multiply them,
+    and push the result back onto the stack
+  }
+  the result is in the top-most item on the stack
+}
+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Bit Set/index.html b/Bit Set/index.html new file mode 100644 index 000000000..7e17b9d0b --- /dev/null +++ b/Bit Set/index.html @@ -0,0 +1,297 @@ + + + Bit Set + + + +

Bit Set

+

A fixed-size sequence of n bits. Also known as bit array or bit vector.

+

To store whether something is true or false you use a Bool. Every programmer knows that... But what if you need to remember whether 10,000 things are true or not?

+

You could make an array of 10,000 booleans but you can also go hardcore and use 10,000 bits instead. That's a lot more compact because 10,000 bits fit in less than 160 Ints on a 64-bit CPU.

+

Since manipulating individual bits is a little tricky, you can use BitSet to hide the dirty work.

+

The code

+

A bit set is simply a wrapper around an array. The array doesn't store individual bits but larger integers called the "words". The main job of BitSet is to map the bits to the right word.

+
public struct BitSet {
+  private(set) public var size: Int
+
+  private let N = 64
+  public typealias Word = UInt64
+  fileprivate(set) public var words: [Word]
+
+  public init(size: Int) {
+    precondition(size > 0)
+    self.size = size
+
+    // Round up the count to the next multiple of 64.
+    let n = (size + (N-1)) / N
+    words = [Word](repeating: 0, count: n)
+  }
+

N is the bit size of the words. It is 64 because we store the bits in a list of unsigned 64-bit integers. (It's fairly easy to change BitSet to use 32-bit words instead.)

+

If you write,

+
var bits = BitSet(size: 140)
+

then the BitSet allocates an array of three words. Each word has 64 bits and therefore three words can hold 192 bits. We only use 140 of those bits so we're wasting a bit of space (but of course we can never use less than a whole word).

+
+

Note: The first entry in the words array is the least-significant word, so these words are stored in little endian order in the array.

+
+

Looking up the bits

+

Most of the operations on BitSet take the index of the bit as a parameter, so it's useful to have a way to find which word contains that bit.

+
  private func indexOf(_ i: Int) -> (Int, Word) {
+    precondition(i >= 0)
+    precondition(i < size)
+    let o = i / N
+    let m = Word(i - o*N)
+    return (o, 1 << m)
+  }
+

The indexOf() function returns the array index of the word, as well as a "mask" that shows exactly where the bit sits inside that word.

+

For example, indexOf(2) returns the tuple (0, 4) because bit 2 is in the first word (index 0). The mask is 4. In binary the mask looks like the following:

+
0010000000000000000000000000000000000000000000000000000000000000
+
+

That 1 points at the second bit in the word.

+
+

Note: Remember that everything is shown in little-endian order, including the bits themselves. Bit 0 is on the left, bit 63 on the right.

+
+

Another example: indexOf(127) returns the tuple (1, 9223372036854775808). It is the last bit of the second word. The mask is:

+
0000000000000000000000000000000000000000000000000000000000000001
+
+

Note that the mask is always 64 bits because we look at the data one word at a time.

+

Setting and getting bits

+

Now that we know where to find a bit, setting it to 1 is easy:

+
  public mutating func set(_ i: Int) {
+    let (j, m) = indexOf(i)
+    words[j] |= m
+  }
+

This looks up the word index and the mask, then performs a bitwise OR between that word and the mask. If the bit was 0 it becomes 1. If it was already set, then it remains set.

+

Clearing the bit -- i.e. changing it to 0 -- is just as easy:

+
  public mutating func clear(_ i: Int) {
+    let (j, m) = indexOf(i)
+    words[j] &= ~m
+  }
+

Instead of a bitwise OR we now do a bitwise AND with the inverse of the mask. So if the mask was 00100000...0, then the inverse is 11011111...1. All the bits are 1, except for the bit we want to set to 0. Due to the way & works, this leaves all other bits alone and only changes that single bit to 0.

+

To see if a bit is set we also use the bitwise AND but without inverting:

+
  public func isSet(_ i: Int) -> Bool {
+    let (j, m) = indexOf(i)
+    return (words[j] & m) != 0
+  }
+

We can add a subscript function to make this all very natural to express:

+
  public subscript(i: Int) -> Bool {
+    get { return isSet(i) }
+    set { if newValue { set(i) } else { clear(i) } }
+  }
+

Now you can write things like:

+
var bits = BitSet(size: 140)
+bits[2] = true
+bits[99] = true
+bits[128] = true
+print(bits)
+

This will print the three words that the 140-bit BitSet uses to store everything:

+
0010000000000000000000000000000000000000000000000000000000000000
+0000000000000000000000000000000000010000000000000000000000000000
+1000000000000000000000000000000000000000000000000000000000000000
+

Something else that's fun to do with bits is flipping them. This changes 0 into 1 and 1 into 0. Here's flip():

+
  public mutating func flip(_ i: Int) -> Bool {
+    let (j, m) = indexOf(i)
+    words[j] ^= m
+    return (words[j] & m) != 0
+  }
+

This uses the remaining bitwise operator, exclusive-OR, to do the flipping. The function also returns the new value of the bit.

+

Ignoring the unused bits

+

A lot of the BitSet functions are quite easy to implement. For example, clearAll(), which resets all the bits to 0:

+
  public mutating func clearAll() {
+    for i in 0..<words.count {
+      words[i] = 0
+    }
+  }
+

There is also setAll() to make all the bits 1. However, this has to deal with a subtle issue.

+
  public mutating func setAll() {
+    for i in 0..<words.count {
+      words[i] = allOnes
+    }
+    clearUnusedBits()
+  }
+

First, we copy ones into all the words in our array. The array is now:

+
1111111111111111111111111111111111111111111111111111111111111111
+1111111111111111111111111111111111111111111111111111111111111111
+1111111111111111111111111111111111111111111111111111111111111111
+

But this is incorrect... Since we don't use most of the last word, we should leave those bits at 0:

+
1111111111111111111111111111111111111111111111111111111111111111
+1111111111111111111111111111111111111111111111111111111111111111
+1111111111110000000000000000000000000000000000000000000000000000
+

Instead of 192 one-bits we now have only 140 one-bits. The fact that the last word may not be completely filled up means that we always have to treat this last word specially.

+

Setting those "leftover" bits to 0 is what the clearUnusedBits() helper function does. If the BitSet's size is not a multiple of N (i.e. 64), then we have to clear out the bits that we're not using. If we don't do this, bitwise operations between two differently sized BitSets will go wrong (an example follows).

+

This uses some advanced bit manipulation, so pay close attention:

+
  private func lastWordMask() -> Word {
+    let diff = words.count*N - size       // 1
+    if diff > 0 {
+      let mask = 1 << Word(63 - diff)     // 2
+      return mask | (mask - 1)            // 3
+    } else {
+      return ~Word()
+    }
+  }
+
+  private mutating func clearUnusedBits() {
+    words[words.count - 1] &= lastWordMask()   // 4
+  }
+

Here's what it does, step-by-step:

+
    +
  1. +

    diff is the number of "leftover" bits. In the above example that is 52 because 3*64 - 140 = 52.

    +
    2. +
  2. +

    Create a mask that is all 0's, except the highest bit that's still valid is a 1. In our example, that would be:

    +

    0000000000010000000000000000000000000000000000000000000000000000

    +
    3. +
  3. +

    Subtract 1 to turn it into:

    +

    1111111111100000000000000000000000000000000000000000000000000000

    +
    4. +
+

and add the high bit back in to get:

+
1111111111110000000000000000000000000000000000000000000000000000
+
+

There are now 12 one-bits in this word because 140 - 2*64 = 12.

+
    +
  1. Finally, turn all the higher bits off. Any leftover bits in the last word are now all 0.
    2. +
+

An example of where this is important is when you combine two BitSets of different sizes. For the sake of illustration, let's take the bitwise OR between two 8-bit values:

+
10001111  size=4
+00100011  size=8
+
+

The first one only uses the first 4 bits; the second one uses 8 bits. The first one should really be 10000000 but let's pretend we forgot to clear out those 1's at the end. Then a bitwise OR between the two results in:

+
10001111  
+00100011  
+-------- OR
+10101111
+
+

That is wrong since two of those 1-bits aren't supposed to be here. The correct way to do it is:

+
10000000       unused bits set to 0 first!
+00100011  
+-------- OR
+10100011
+
+

Here's how the | operator is implemented:

+
public func |(lhs: BitSet, rhs: BitSet) -> BitSet {
+  var out = copyLargest(lhs, rhs)
+  let n = min(lhs.words.count, rhs.words.count)
+  for i in 0..<n {
+    out.words[i] = lhs.words[i] | rhs.words[i]
+  }
+  return out
+}
+

Note that we | entire words together, not individual bits. That would be way too slow! We also need to do some extra work if the left-hand side and right-hand side have a different number of bits: we copy the largest of the two BitSets into the out variable and then combine it with the words from the smaller BitSet.

+

Example:

+
var a = BitSet(size: 4)
+a.setAll()
+a[1] = false
+a[2] = false
+a[3] = false
+print(a)
+
+var b = BitSet(size: 8)
+b[2] = true
+b[6] = true
+b[7] = true
+print(b)
+
+let c = a | b
+print(c)        // 1010001100000000...0
+

Bitwise AND (&), exclusive-OR (^), and inversion (~) are implemented in a similar manner.

+

Counting the number of 1-bits

+

To count the number of bits that are set to 1 we could scan through the entire array -- an O(n) operation -- but there's a more clever method:

+
  public var cardinality: Int {
+    var count = 0
+    for var x in words {
+      while x != 0 {
+        let y = x & ~(x - 1)  // find lowest 1-bit
+        x = x ^ y             // and erase it
+        ++count
+      }
+    }
+    return count
+  }
+

When you write x & ~(x - 1), it gives you a new value with only a single bit set. This is the lowest bit that is one. For example take this 8-bit value (again, I'm showing this with the least significant bit on the left):

+
00101101
+
+

First we subtract 1 to get:

+
11001101
+
+

Then we invert it, flipping all the bits:

+
00110010
+
+

And take the bitwise AND with the original value:

+
00101101
+00110010
+-------- AND
+00100000
+
+

The only value they have in common is the lowest (or least significant) 1-bit. Then we erase that from the original value using exclusive-OR:

+
00101101
+00100000
+-------- XOR
+00001101
+
+

This is the original value but with the lowest 1-bit removed.

+

We keep repeating this process until the value consists of all zeros. The time complexity is O(s) where s is the number of 1-bits.

+

Bit Shift Operations

+

Bit shifts are a common and very useful mechanism when dealing with bitsets. Here is the right-shift function:

+
public func >> (lhs: BitSet, numBitsRight: Int) -> BitSet {
+    var out = lhs
+    let offset = numBitsRight / lhs.N
+    let shift = numBitsRight % lhs.N
+    for i in 0..<lhs.words.count {
+        out.words[i] = 0
+        
+        if (i + offset < lhs.words.count) {
+            out.words[i] = lhs.words[i + offset] >> shift
+        }
+        
+        if (i + offset + 1 < lhs.words.count) {
+            out.words[i] |= lhs.words[i + offset + 1] << (lhs.N - shift)
+        }
+    }
+
+    out.clearUnusedBits()
+    return out
+}
+
+

Let's start with this line:

+
for i in 0..<lhs.words.count {
+

This indicates our strategy: we want to go through each word of the result and assign the correct bits.

+

The two internal if-statements inside this loop are assigning some bits from one place in the source number and bitwise OR-ing them with some bits from a second place in the source number. The key insight here is that the bits for any one word of the result comes from at most two source words in the input. So the only remaining trick is to calculate which ones. For this, we need these two numbers:

+
let offset = numBitsRight / lhs.N
+let shift = numBitsRight % lhs.N
+

Offset gives us how many words away from the source word we start getting our bits (with the remainder coming from it's neighbour). Shift gives us how many bits we need to shift within that word. Note that both of these are calcuated using the word size lhs.N.

+

All that's left if a little bit of protection against reading outside the bounds of the input. So these two if conditions protect against that:

+
if (i + offset < lhs.words.count) {
+

and

+
if (i + offset + 1 < lhs.words.count) {
+

Let's work through an example. Suppose our word length has been reduced to 8 bits, and we'd like to right-shift the following number by 10 bits:

+
01000010 11000000 00011100      >> 10
+
+

I've grouped each part of the number by word to make it easier to see what happens. The for-loop goes from least significant word to most significant. So for index zero we're want to know what bits will make up our least significant word. Let's calculate our offset and shift values:

+
offset = 10 / 8 = 1     (remember this is integer division)
+shift = 10 % 8 = 2
+
+

So we consult the word at offset 1 to get some of our bits:

+
11000000 >> 2 = 00110000
+
+

And we get the rest of them from the word one further away:

+
01000010 << (8 - 2) = 10000000
+
+

And we bitwise OR these together to get our least significant term

+
00110000
+10000000
+-------- OR
+10110000
+
+

We repeat this for the 2nd least significant term and obtain:

+
00010000
+
+

The last term can't get any bits because they are past the end of our number so those are all zeros. Our result is:

+
00000000 00010000 10110000
+
+

See also

+

Bit Twiddling Hacks

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Bloom Filter/index.html b/Bloom Filter/index.html new file mode 100644 index 000000000..7b4c6c43e --- /dev/null +++ b/Bloom Filter/index.html @@ -0,0 +1,87 @@ + + + Bloom Filter + + + +

Bloom Filter

+

Introduction

+

A Bloom Filter is a space-efficient data structure that tells you whether or not an element is present in a set.

+

This is a probabilistic data structure: a query to a Bloom filter either returns false, meaning the element is definitely not in the set, or true, meaning that the element might be in the set.

+

There is a small probability of false positives, where the element isn't actually in the set even though the query returned true. But there will never any false negatives: you're guaranteed that if the query returns false, then the element really isn't in the set.

+

So a Bloom Filter tells you, "definitely not" or "probably yes".

+

At first, this may not seem too useful. However, it's important in applications like cache filtering and data synchronization.

+

An advantage of the Bloom Filter over a hash table is that the former maintains constant memory usage and constant-time insert and search. For sets with a large number of elements, the performance difference between a hash table and a Bloom Filter is significant, and it is a viable option if you do not need the guarantee of no false positives.

+
+

Note: Unlike a hash table, the Bloom Filter does not store the actual objects. It just remembers what objects you’ve seen (with a degree of uncertainty) and which ones you haven’t.

+
+

Inserting objects into the set

+

A Bloom Filter is essentially a fixed-length bit vector, an array of bits. When we insert objects, we set some of these bits to 1, and when we query for objects we check if certain bits are 0 or 1. Both operations use hash functions.

+

To insert an element in the filter, the element is hashed with several different hash functions. Each hash function returns a value that we map to an index in the array. We then set the bits at these indices to 1 or true.

+

For example, let's say this is our array of bits. We have 17 bits and initially they are all 0 or false:

+
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
+
+

Now we want to insert the string "Hello world!" into the Bloom Filter. We apply two hash functions to this string. The first one gives the value 1999532104120917762. We map this hash value to an index into our array by taking the modulo of the array length: 1999532104120917762 % 17 = 4. This means we set the bit at index 4 to 1 or true:

+
[ 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
+
+

Then we hash the original string again but this time with a different hash function. It gives the hash value 9211818684948223801. Modulo 17 that is 12, and we set the bit at index 12 to 1 as well:

+
[ 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 ]
+
+

These two 1-bits are enough to tell the Bloom Filter that it now contains the string "Hello world!". Of course, it doesn't contain the actual string, so you can't ask the Bloom Filter, "give me a list of all the objects you contain". All it has is a bunch of ones and zeros.

+

Querying the set

+

Querying, similarly to inserting, is accomplished by first hashing the expected value, which gives several array indices, and then checking to see if all of the bits at those indices are 1. If even one of the bits is not 1, the element could not have been inserted and the query returns false. If all the bits are 1, the query returns true.

+

For example, if we query for the string "Hello WORLD", then the first hash function returns 5383892684077141175, which modulo 17 is 12. That bit is 1. But the second hash function gives 5625257205398334446, which maps to array index 9. That bit is 0. This means the string "Hello WORLD" is not in the filter and the query returns false.

+

The fact that the first hash function mapped to a 1 bit is a coincidence (it has nothing to do with the fact that both strings start with "Hello "). Too many such coincidences can lead to "collisions". If there are collisions, the query may erroneously return true even though the element was not inserted -- bringing about the issue with false positives mentioned earlier.

+

Let's say we insert some other element, "Bloom Filterz", which sets bits 7 and 9. Now the array looks like this:

+
[ 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0 ]
+
+

If you query for "Hello WORLD" again, the filter sees that bit 12 is true and bit 9 is now true as well. It reports that "Hello WORLD" is indeed present in the set, even though it isn't... because we never inserted that particular string. It's a false positive. This example shows why a Bloom Filter will never say, "definitely yes", only "probably yes".

+

You can fix such issues by using an array with more bits and using additional hash functions. Of course, the more hash functions you use the slower the Bloom Filter will be. So you have to strike a balance.

+

Deletion is not possible with a Bloom Filter, since any one bit might belong to multiple elements. Once you add an element, it's in there for good.

+

Performance of a Bloom Filter is O(k) where k is the number of hashing functions.

+

The code

+

The code is quite straightforward. The internal bit array is set to a fixed length on initialization, which cannot be mutated once it is initialized.

+
public init(size: Int = 1024, hashFunctions: [(T) -> Int]) {
+	self.array = [Bool](repeating: false, count: size)
+  self.hashFunctions = hashFunctions
+}
+

Several hash functions should be specified at initialization. Which hash functions you use will depend on the datatypes of the elements you'll be adding to the set. You can see some examples in the playground and the tests -- the djb2 and sdbm hash functions for strings.

+

Insertion just flips the required bits to true:

+
public func insert(_ element: T) {
+  for hashValue in computeHashes(element) {
+    array[hashValue] = true
+  }
+}
+

This uses the computeHashes() function, which loops through the specified hashFunctions and returns an array of indices:

+
private func computeHashes(_ value: T) -> [Int] {
+  return hashFunctions.map() { hashFunc in abs(hashFunc(value) % array.count) }
+}
+

And querying checks to make sure the bits at the hashed values are true:

+
public func query(_ value: T) -> Bool {
+  let hashValues = computeHashes(value)
+  let results = hashValues.map() { hashValue in array[hashValue] }
+	let exists = results.reduce(true, { $0 && $1 })
+  return exists
+}
+

If you're coming from another imperative language, you might notice the unusual syntax in the exists assignment. Swift makes use of functional paradigms when it makes code more consise and readable, and in this case reduce is a much more consise way to check if all the required bits are true than a for loop.

+

Another approach

+

In the previous section, you learnt about how using multiple different hash functions can help reduce the chance of collisions in the bloom filter. However, good hash functions are difficult to design. A simple alternative to multiple hash functions is to use a set of random numbers.

+

As an example, let's say a bloom filter wants to hash each element 15 times during insertion. Instead of using 15 different hash functions, you can rely on just one hash function. The hash value can then be combined with 15 different values to form the indices for flipping. This bloom filter would initialize a set of 15 random numbers ahead of time and use these values during each insertion.

+
hash("Hello world!") >> hash(987654321) // would flip bit 8
+hash("Hello world!") >> hash(123456789) // would flip bit 2
+
+

Since Swift 4.2, Hasher is now included in the Standard library, which is designed to reduce multiple hashes to a single hash in an efficient manner. This makes combining the hashes trivial.

+
private func computeHashes(_ value: T) -> [Int] {
+  return randomSeeds.map() { seed in
+		let hasher = Hasher()
+		hasher.combine(seed)
+		hasher.combine(value)
+		let hashValue = hasher.finalize()
+		return abs(hashValue % array.count)
+	}
+}
+
+

If you want to learn more about this approach, you can read about the Hasher documentation or Soroush Khanlou's Swift 4.2 Bloom filter implementation.

+

Written for Swift Algorithm Club by Jamil Dhanani. Edited by Matthijs Hollemans. Updated by Bruno Scheele.

+ + diff --git a/Bounded Priority Queue/index.html b/Bounded Priority Queue/index.html new file mode 100644 index 000000000..d024a25eb --- /dev/null +++ b/Bounded Priority Queue/index.html @@ -0,0 +1,138 @@ + + + Bounded Priority Queue + + + +

Bounded Priority queue

+

A bounded priority queue is similar to a regular priority queue, except that there is a fixed upper bound on the number of elements that can be stored. When a new element is added to the queue while the queue is at capacity, the element with the highest priority value is ejected from the queue.

+

Example

+

Suppose we have a bounded-priority-queue with maximum size 5 that has the following values and priorities:

+
Value:    [ A,   B,   C,    D,    E   ]
+Priority: [ 4.6, 3.2, 1.33, 0.25, 0.1 ]
+
+

Here, we consider the object with the highest priority value to be the most important (so this is a max-priority queue). The larger the priority value, the more we care about the object. So A is more important than B, B is more important than C, and so on.

+

Now we want to insert the element F with priority 0.4 into this bounded priority queue. Because the queue has maximum size 5, this will insert the element F but then evict the lowest-priority element (E), yielding the updated queue:

+
Value:    [ A,   B,   C,    F,   D    ]
+Priority: [ 4.6, 3.2, 1.33, 0.4, 0.25 ]
+
+

F is inserted between C and D because of its priority value. It's less important than C but more important than D.

+

Suppose that we wish to insert the element G with priority 0.1 into this BPQ. Because G's priority value is less than the minimum-priority element in the queue, upon inserting G it will immediately be evicted. In other words, inserting an element into a BPQ with priority less than the minimum-priority element of the BPQ has no effect.

+

Implementation

+

While a heap may be a really simple implementation for a priority queue, a sorted linked list allows for O(k) insertion and O(1) deletion, where k is the bounding number of elements.

+

Here's how you could implement it in Swift:

+
public class BoundedPriorityQueue<T: Comparable> {
+  private typealias Node = LinkedListNode<T>
+
+  private(set) public var count = 0
+  fileprivate var head: Node?
+  private var tail: Node?
+  private var maxElements: Int
+
+  public init(maxElements: Int) {
+    self.maxElements = maxElements
+  }
+
+  public var isEmpty: Bool {
+    return count == 0
+  }
+
+  public func peek() -> T? {
+    return head?.value
+  }
+

The BoundedPriorityQueue class contains a doubly linked list of LinkedListNode objects. Nothing special here yet. The fun stuff happens in the enqueue() method:

+
public func enqueue(_ value: T) {
+  if let node = insert(value, after: findInsertionPoint(value)) {
+    // If the newly inserted node is the last one in the list, then update
+    // the tail pointer.
+    if node.next == nil {
+      tail = node
+    }
+
+    // If the queue is full, then remove an element from the back.
+    count += 1
+    if count > maxElements {
+      removeLeastImportantElement()
+    }
+  }
+}
+
+private func insert(_ value: T, after: Node?) -> Node? {
+  if let previous = after {
+
+    // If the queue is full and we have to insert at the end of the list,
+    // then there's no reason to insert the new value.
+    if count == maxElements && previous.next == nil {
+      print("Queue is full and priority of new object is too small")
+      return nil
+    }
+
+    // Put the new node in between previous and previous.next (if exists).
+    let node = Node(value: value)
+    node.next = previous.next
+    previous.next?.previous = node
+    previous.next = node
+    node.previous = previous
+    return node
+
+  } else if let first = head {
+    // Have to insert at the head, so shift the existing head up once place.
+    head = Node(value: value)
+    head!.next = first
+    first.previous = head
+    return head
+
+  } else {
+    // This is the very first item in the queue.
+    head = Node(value: value)
+    return head
+  }
+}
+
+/* Find the node after which to insert the new value. If this returns nil,
+   the new value should be inserted at the head of the list. */
+private func findInsertionPoint(_ value: T) -> Node? {
+  var node = head
+  var prev: Node? = nil
+
+  while let current = node where value < current.value {
+    prev = node
+    node = current.next
+  }
+  return prev
+}
+
+private func removeLeastImportantElement() {
+  if let last = tail {
+    tail = last.previous
+    tail?.next = nil
+    count -= 1
+  }
+
+  // Note: Instead of using a tail pointer, we could just scan from the new
+  // node until the end. Then nodes also don't need a previous pointer. But
+  // this is much slower on large lists.
+}
+

We first check if the queue already has the maximum number of elements. If so, and the new priority value is less than the tail element's priority value, then there is no room for this new element and we return without inserting it.

+

If the new value is acceptable, then we search through the list to find the proper insertion location and update the next and previous pointers.

+

Lastly, if the queue has now reached the maximum number of elements, then we dequeue() the one with the largest priority value.

+

By keeping the most important element at the front of the list, it makes dequeueing very easy:

+
public func dequeue() -> T? {
+  if let first = head {
+    count -= 1
+    if count == 0 {
+      head = nil
+      tail = nil
+    } else {
+      head = first.next
+      head!.previous = nil
+    }
+    return first.value
+  } else {
+    return nil
+  }
+}
+

This simply removes the head element from the list and returns it.

+

Written for Swift Algorithm Club by John Gill and Matthijs Hollemans

+ + diff --git a/Boyer-Moore-Horspool/index.html b/Boyer-Moore-Horspool/index.html new file mode 100644 index 000000000..c5bae75f4 --- /dev/null +++ b/Boyer-Moore-Horspool/index.html @@ -0,0 +1,206 @@ + + + Boyer-Moore-Horspool + + + +

Boyer-Moore String Search

+
+

This topic has been tutorialized here

+
+

Goal: Write a string search algorithm in pure Swift without importing Foundation or using NSString's rangeOfString() method.

+

In other words, we want to implement an indexOf(pattern: String) extension on String that returns the String.Index of the first occurrence of the search pattern, or nil if the pattern could not be found inside the string.

+

For example:

+
// Input:
+let s = "Hello, World"
+s.indexOf(pattern: "World")
+
+// Output:
+<String.Index?> 7
+
+// Input:
+let animals = "🐶🐔🐷🐮🐱"
+animals.indexOf(pattern: "🐮")
+
+// Output:
+<String.Index?> 6
+
+

Note: The index of the cow is 6, not 3 as you might expect, because the string uses more storage per character for emoji. The actual value of the String.Index is not so important, just that it points at the right character in the string.

+
+

The brute-force approach works OK, but it's not very efficient, especially on large chunks of text. As it turns out, you don't need to look at every character from the source string -- you can often skip ahead multiple characters.

+

The skip-ahead algorithm is called Boyer-Moore and it has been around for a long time. It is considered the benchmark for all string search algorithms.

+

Here's how you could write it in Swift:

+
extension String {
+    func index(of pattern: String) -> Index? {
+        // Cache the length of the search pattern because we're going to
+        // use it a few times and it's expensive to calculate.
+        let patternLength = pattern.count
+        guard patternLength > 0, patternLength <= count else { return nil }
+
+        // Make the skip table. This table determines how far we skip ahead
+        // when a character from the pattern is found.
+        var skipTable = [Character: Int]()
+        for (i, c) in pattern.enumerated() {
+            skipTable[c] = patternLength - i - 1
+        }
+
+        // This points at the last character in the pattern.
+        let p = pattern.index(before: pattern.endIndex)
+        let lastChar = pattern[p]
+
+        // The pattern is scanned right-to-left, so skip ahead in the string by
+        // the length of the pattern. (Minus 1 because startIndex already points
+        // at the first character in the source string.)
+        var i = index(startIndex, offsetBy: patternLength - 1)
+
+        // This is a helper function that steps backwards through both strings
+        // until we find a character that doesn’t match, or until we’ve reached
+        // the beginning of the pattern.
+        func backwards() -> Index? {
+            var q = p
+            var j = i
+            while q > pattern.startIndex {
+                j = index(before: j)
+                q = index(before: q)
+                if self[j] != pattern[q] { return nil }
+            }
+            return j
+        }
+
+        // The main loop. Keep going until the end of the string is reached.
+        while i < endIndex {
+            let c = self[i]
+
+            // Does the current character match the last character from the pattern?
+            if c == lastChar {
+
+                // There is a possible match. Do a brute-force search backwards.
+                if let k = backwards() { return k }
+
+                // If no match, we can only safely skip one character ahead.
+                i = index(after: i)
+            } else {
+                // The characters are not equal, so skip ahead. The amount to skip is
+                // determined by the skip table. If the character is not present in the
+                // pattern, we can skip ahead by the full pattern length. However, if
+                // the character *is* present in the pattern, there may be a match up
+                // ahead and we can't skip as far.
+                i = index(i, offsetBy: skipTable[c] ?? patternLength, limitedBy: endIndex) ?? endIndex
+            }
+        }
+        return nil
+    }
+}
+

The algorithm works as follows. You line up the search pattern with the source string and see what character from the string matches the last character of the search pattern:

+
source string:  Hello, World
+search pattern: World
+                    ^
+
+

There are three possibilities:

+
    +
  1. +

    The two characters are equal. You've found a possible match.

    +
    2. +
  2. +

    The characters are not equal, but the source character does appear in the search pattern elsewhere.

    +
    3. +
  3. +

    The source character does not appear in the search pattern at all.

    +
    4. +
+

In the example, the characters o and d do not match, but o does appear in the search pattern. That means we can skip ahead several positions:

+
source string:  Hello, World
+search pattern:    World
+                       ^
+
+

Note how the two o characters line up now. Again you compare the last character of the search pattern with the search text: W vs d. These are not equal but the W does appear in the pattern. So skip ahead again to line up those two W characters:

+
source string:  Hello, World
+search pattern:        World
+                           ^
+
+

This time the two characters are equal and there is a possible match. To verify the match you do a brute-force search, but backwards, from the end of the search pattern to the beginning. And that's all there is to it.

+

The amount to skip ahead at any given time is determined by the "skip table", which is a dictionary of all the characters in the search pattern and the amount to skip by. The skip table in the example looks like:

+
W: 4
+o: 3
+r: 2
+l: 1
+d: 0
+
+

The closer a character is to the end of the pattern, the smaller the skip amount. If a character appears more than once in the pattern, the one nearest to the end of the pattern determines the skip value for that character.

+
+

Note: If the search pattern consists of only a few characters, it's faster to do a brute-force search. There's a trade-off between the time it takes to build the skip table and doing brute-force for short patterns.

+
+

Credits: This code is based on the article "Faster String Searches" by Costas Menico from Dr Dobb's magazine, July 1989 -- Yes, 1989! Sometimes it's useful to keep those old magazines around.

+

See also: a detailed analysis of the algorithm.

+

Boyer-Moore-Horspool algorithm

+

A variation on the above algorithm is the Boyer-Moore-Horspool algorithm.

+

Like the regular Boyer-Moore algorithm, it uses the skipTable to skip ahead a number of characters. The difference is in how we check partial matches. In the above version, if a partial match is found but it's not a complete match, we skip ahead by just one character. In this revised version, we also use the skip table in that situation.

+

Here's an implementation of the Boyer-Moore-Horspool algorithm:

+
extension String {
+    func index(of pattern: String) -> Index? {
+        // Cache the length of the search pattern because we're going to
+        // use it a few times and it's expensive to calculate.
+        let patternLength = pattern.count
+        guard patternLength > 0, patternLength <= characters.count else { return nil }
+
+        // Make the skip table. This table determines how far we skip ahead
+        // when a character from the pattern is found.
+        var skipTable = [Character: Int]()
+        for (i, c) in pattern.enumerated() {
+            skipTable[c] = patternLength - i - 1
+        }
+
+        // This points at the last character in the pattern.
+        let p = pattern.index(before: pattern.endIndex)
+        let lastChar = pattern[p]
+
+        // The pattern is scanned right-to-left, so skip ahead in the string by
+        // the length of the pattern. (Minus 1 because startIndex already points
+        // at the first character in the source string.)
+        var i = index(startIndex, offsetBy: patternLength - 1)
+
+        // This is a helper function that steps backwards through both strings
+        // until we find a character that doesn’t match, or until we’ve reached
+        // the beginning of the pattern.
+        func backwards() -> Index? {
+            var q = p
+            var j = i
+            while q > pattern.startIndex {
+                j = index(before: j)
+                q = index(before: q)
+                if self[j] != pattern[q] { return nil }
+            }
+            return j
+        }
+
+        // The main loop. Keep going until the end of the string is reached.
+        while i < endIndex {
+            let c = self[i]
+
+            // Does the current character match the last character from the pattern?
+            if c == lastChar {
+
+                // There is a possible match. Do a brute-force search backwards.
+                if let k = backwards() { return k }
+
+                // Ensure to jump at least one character (this is needed because the first
+                // character is in the skipTable, and `skipTable[lastChar] = 0`)
+                let jumpOffset = max(skipTable[c] ?? patternLength, 1)
+                i = index(i, offsetBy: jumpOffset, limitedBy: endIndex) ?? endIndex
+            } else {
+                // The characters are not equal, so skip ahead. The amount to skip is
+                // determined by the skip table. If the character is not present in the
+                // pattern, we can skip ahead by the full pattern length. However, if
+                // the character *is* present in the pattern, there may be a match up
+                // ahead and we can't skip as far.
+                i = index(i, offsetBy: skipTable[c] ?? patternLength, limitedBy: endIndex) ?? endIndex
+            }
+        }
+        return nil
+    }
+}
+

In practice, the Horspool version of the algorithm tends to perform a little better than the original. However, it depends on the tradeoffs you're willing to make.

+

Credits: This code is based on the paper: R. N. Horspool (1980). "Practical fast searching in strings". Software - Practice & Experience 10 (6): 501–506.

+

Written for Swift Algorithm Club by Matthijs Hollemans, updated by Andreas Neusüß, Matías Mazzei.

+ + diff --git a/Breadth-First Search/index.html b/Breadth-First Search/index.html new file mode 100644 index 000000000..2ae2d4621 --- /dev/null +++ b/Breadth-First Search/index.html @@ -0,0 +1,104 @@ + + + Breadth-First Search + + + +

Breadth-First Search

+
+

This topic has been tutorialized here

+
+

Breadth-first search (BFS) is an algorithm for traversing or searching tree or graph data structures. It starts at a source node and explores the immediate neighbor nodes first, before moving to the next level neighbors.

+

Breadth-first search can be used on both directed and undirected graphs.

+

Animated example

+

Here's how breadth-first search works on a graph:

+

Animated example of a breadth-first search

+

When we visit a node, we color it black. We also put its neighbor nodes into a queue. In the animation the nodes that are enqueued but not visited yet are shown in gray.

+

Let's follow the animated example. We start with the source node A and add it to a queue. In the animation this is shown as node A becoming gray.

+
queue.enqueue(A)
+

The queue is now [ A ]. The idea is that, as long as there are nodes in the queue, we visit the node that's at the front of the queue, and enqueue its immediate neighbor nodes if they have not been visited yet.

+

To start traversing the graph, we pull the first node off the queue, A, and color it black. Then we enqueue its two neighbor nodes B and C. This colors them gray.

+
queue.dequeue()   // A
+queue.enqueue(B)
+queue.enqueue(C)
+

The queue is now [ B, C ]. We dequeue B, and enqueue B's neighbor nodes D and E.

+
queue.dequeue()   // B
+queue.enqueue(D)
+queue.enqueue(E)
+

The queue is now [ C, D, E ]. Dequeue C, and enqueue C's neighbor nodes F and G.

+
queue.dequeue()   // C
+queue.enqueue(F)
+queue.enqueue(G)
+

The queue is now [ D, E, F, G ]. Dequeue D, which has no neighbor nodes.

+
queue.dequeue()   // D
+

The queue is now [ E, F, G ]. Dequeue E and enqueue its single neighbor node H. Note that B is also a neighbor for E but we've already visited B, so we're not adding it to the queue again.

+
queue.dequeue()   // E
+queue.enqueue(H)
+

The queue is now [ F, G, H ]. Dequeue F, which has no unvisited neighbor nodes.

+
queue.dequeue()   // F
+

The queue is now [ G, H ]. Dequeue G, which has no unvisited neighbor nodes.

+
queue.dequeue()   // G
+

The queue is now [ H ]. Dequeue H, which has no unvisited neighbor nodes.

+
queue.dequeue()   // H
+

The queue is now empty, meaning that all nodes have been explored. The order in which the nodes were explored is A, B, C, D, E, F, G, H.

+

We can show this as a tree:

+

The BFS tree

+

The parent of a node is the one that "discovered" that node. The root of the tree is the node you started the breadth-first search from.

+

For an unweighted graph, this tree defines a shortest path from the starting node to every other node in the tree. So breadth-first search is one way to find the shortest path between two nodes in a graph.

+

The code

+

Simple implementation of breadth-first search using a queue:

+
func breadthFirstSearch(_ graph: Graph, source: Node) -> [String] {
+  var queue = Queue<Node>()
+  queue.enqueue(source)
+
+  var nodesExplored = [source.label]
+  source.visited = true
+
+  while let node = queue.dequeue() {
+    for edge in node.neighbors {
+      let neighborNode = edge.neighbor
+      if !neighborNode.visited {
+        queue.enqueue(neighborNode)
+        neighborNode.visited = true
+        nodesExplored.append(neighborNode.label)
+      }
+    }
+  }
+
+  return nodesExplored
+}
+

While there are nodes in the queue, we visit the first one and then enqueue its immediate neighbors if they haven't been visited yet.

+

Put this code in a playground and test it like so:

+
let graph = Graph()
+
+let nodeA = graph.addNode("a")
+let nodeB = graph.addNode("b")
+let nodeC = graph.addNode("c")
+let nodeD = graph.addNode("d")
+let nodeE = graph.addNode("e")
+let nodeF = graph.addNode("f")
+let nodeG = graph.addNode("g")
+let nodeH = graph.addNode("h")
+
+graph.addEdge(nodeA, neighbor: nodeB)
+graph.addEdge(nodeA, neighbor: nodeC)
+graph.addEdge(nodeB, neighbor: nodeD)
+graph.addEdge(nodeB, neighbor: nodeE)
+graph.addEdge(nodeC, neighbor: nodeF)
+graph.addEdge(nodeC, neighbor: nodeG)
+graph.addEdge(nodeE, neighbor: nodeH)
+graph.addEdge(nodeE, neighbor: nodeF)
+graph.addEdge(nodeF, neighbor: nodeG)
+
+let nodesExplored = breadthFirstSearch(graph, source: nodeA)
+print(nodesExplored)
+

This will output: ["a", "b", "c", "d", "e", "f", "g", "h"]

+

What is BFS good for?

+

Breadth-first search can be used to solve many problems. A small selection:

+ +

Written by Chris Pilcher and Matthijs Hollemans

+ + diff --git a/Brute-Force String Search/index.html b/Brute-Force String Search/index.html new file mode 100644 index 000000000..3f247ae25 --- /dev/null +++ b/Brute-Force String Search/index.html @@ -0,0 +1,52 @@ + + + Brute-Force String Search + + + +

Brute-Force String Search

+

How would you go about writing a string search algorithm in pure Swift if you were not allowed to import Foundation and could not use NSString's rangeOfString() method?

+

The goal is to implement an indexOf(pattern: String) extension on String that returns the String.Index of the first occurrence of the search pattern, or nil if the pattern could not be found inside the string.

+

For example:

+
// Input: 
+let s = "Hello, World"
+s.indexOf("World")
+
+// Output:
+<String.Index?> 7
+
+// Input:
+let animals = "🐶🐔🐷🐮🐱"
+animals.indexOf("🐮")
+
+// Output:
+<String.Index?> 6
+
+

Note: The index of the cow is 6, not 3 as you might expect, because the string uses more storage per character for emoji. The actual value of the String.Index is not so important, just that it points at the right character in the string.

+
+

Here is a brute-force solution:

+
extension String {
+  func indexOf(_ pattern: String) -> String.Index? {
+    for i in self.characters.indices {
+        var j = i
+        var found = true
+        for p in pattern.characters.indices{
+            if j == self.characters.endIndex || self[j] != pattern[p] {
+                found = false
+                break
+            } else {
+                j = self.characters.index(after: j)
+            }
+        }
+        if found {
+            return i
+        }
+    }
+    return nil
+  }
+}
+

This looks at each character in the source string in turn. If the character equals the first character of the search pattern, then the inner loop checks whether the rest of the pattern matches. If no match is found, the outer loop continues where it left off. This repeats until a complete match is found or the end of the source string is reached.

+

The brute-force approach works OK, but it's not very efficient (or pretty). It should work fine on small strings, though. For a smarter algorithm that works better with large chunks of text, check out Boyer-Moore string search.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Bubble Sort/index.html b/Bubble Sort/index.html new file mode 100644 index 000000000..982f40fbd --- /dev/null +++ b/Bubble Sort/index.html @@ -0,0 +1,92 @@ + + + Bubble Sort + + + +

Bubble Sort

+

Bubble sort is a sorting algorithm that is implemented by starting in the beginning of the array and swapping the first two elements only if the first element is greater than the second element. This comparison is then moved onto the next pair and so on and so forth. This is done until the array is completely sorted. The smaller items slowly “bubble” up to the beginning of the array. Sometimes this algorithm is refered as Sinking sort, due to the larger, or heavier elements sinking to the end of the array.

+
Runtime:
+ +
Memory:
+ +

Implementation:

+

The implementation will not be shown as the average and worst runtimes show that this is a very inefficient algorithm. However, having a grasp of the concept will help you understand the basics of simple sorting algorithms.

+

Bubble sort is a very simple sorting algorithm, it consists in comparing pairs of adjacent elements in the array, if the first element is larger, swap them, otherwise, you do nothing and go for the next comparison.
+This is accomplished by looking through the array n times, n being the amount of elements in the array.

+

animated gif of the bubble sort algorithm

+

This GIF shows a inverted implementation than

+

Example

+

Let us take the array [5, 1, 4, 2, 8], and sort the array from lowest number to greatest number using bubble sort. In each step, elements written in bold are being compared. Three passes will be required.

+
First Pass
+

[ 5 1 4 2 8 ] -> [ 1 5 4 2 8 ], Here, algorithm compares the first two elements, and swaps since 5 > 1.

+

[ 1 5 4 2 8 ] -> [ 1 4 5 2 8 ], Swap since 5 > 4

+

[ 1 4 5 2 8 ] -> [ 1 4 2 5 8 ], Swap since 5 > 2

+

[ 1 4 2 5 8 ] -> [ 1 4 2 5 8 ], Now, since these elements are already in order (8 > 5), algorithm does not swap them.

+
Second Pass
+

[ 1 4 2 5 8 ] -> [ 1 4 2 5 8 ]

+

[ 1 4 2 5 8 ] -> [ 1 2 4 5 8 ], Swap since 4 > 2

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]
+Now, the array is already sorted, but the algorithm does not know if it is completed. The algorithm needs one whole pass without any swap to know it is sorted.

+
Third Pass
+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

This is the same for the forth and fifth passes.

+

Code

+
for i in 0..<array.count {
+  for j in 1..<array.count {
+    if array[j] < array[j-1] {
+      let tmp = array[j-1]
+      array[j-1] = array[j]
+      array[j] = tmp
+    }
+  }
+}
+return array
+

Optimization

+

The bubble sort algorithm can be easily optimized by observing that the n-th pass finds the n-th largest element and puts it into its final place. So, the inner loop can avoid looking at the last n-1 items when running for the n-th time:

+
for i in 0..<array.count {
+  for j in 1..<array.count - i {
+    if array[j] < array[j-1] {
+      let tmp = array[j-1]
+      array[j-1] = array[j]
+      array[j] = tmp
+    }
+  }
+}
+return array
+

The only change made was on the second line, changing the interval from 1..<array.count to 1..<array.count - i, effectively cutting the number of comparisons by half.

+

The ordering with the optimized code would look something like this for the array [5, 1, 4, 2, 8]:

+
First Pass
+

[ 5 1 4 2 8 ] -> [ 1 5 4 2 8 ], Swaps since 5 > 1

+

[ 1 5 4 2 8 ] -> [ 1 4 5 2 8 ], Swap since 5 > 4

+

[ 1 4 5 2 8 ] -> [ 1 4 2 5 8 ], Swap since 5 > 2

+

[ 1 4 2 5 8 ] -> [ 1 4 2 5 8 ], Now, since these elements are already in order (8 > 5), algorithm does not swap them.

+

by the end of the first pass, the last element is guaranteed to be the largest

+
Second Pass
+

[ 1 4 2 5 8 ] -> [ 1 4 2 5 8 ]

+

[ 1 4 2 5 8 ] -> [ 1 2 4 5 8 ], Swap since 4 > 2

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ], As the first loop has occured once, the inner loop stops here, not comparing 5 with 8

+
Third Pass
+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ] again, stoping one comparison short

+
Fourth Pass
+

[ 1 2 4 5 8 ] -> [ 1 2 4 5 8 ]

+

There is no Fifth pass

+

Conclusion

+

Even with the proposed optimizations, this is still a terribly inefficient sorting algorithm. A good alternative is Merge Sort, that not only is better performing, has a similar degree of dificulty to implement.

+

Updated for the Swift Algorithm Club by Julio Brazil

+
Supporting Links
+

Code Pumpkin
+Wikipedia
+GeeksforGeeks

+ + diff --git a/Bucket Sort/index.html b/Bucket Sort/index.html new file mode 100644 index 000000000..8f1d36bb7 --- /dev/null +++ b/Bucket Sort/index.html @@ -0,0 +1,217 @@ + + + Bucket Sort + + + +

Bucket Sort

+

Bucket Sort, also known as Bin Sort, is a distributed sorting algorithm, which sort elements from an array by performing these steps:

+
    +
  1. Distribute the elements into buckets or bins.
    2. +
  2. Sort each bucket individually.
    3. +
  3. Merge the buckets in order to produce a sorted array as the result.
    4. +
+

See the algorithm in action here and here.

+

The performance for execution time is:

+ + + + + + + + + + + + + + + + + + + + + +
CasePerformance
WorstO(n^2)
BestOmega(n + k)
AverageTheta(n + k)
+

Where n = the number of elements and k is the number of buckets.

+

In the best case, the algorithm distributes the elements uniformly between buckets, a few elements are placed on each bucket and sorting the buckets is O(1). Rearranging the elements is one more run through the initial list.

+

In the worst case, the elements are sent all to the same bucket, making the process take O(n^2).

+

Pseudocode

+

A pseudocode of the algorithm can be as follows:

+
function bucketSort(array, n) is
+    buckets ← new array of n empty lists
+    for i = 0 to (length(array)-1) do
+        insert array[i] into buckets[msbits(array[i], k)]
+    for i = 0 to n - 1 do
+        nextSort(buckets[i]);
+    return the concatenation of buckets[0], ...., buckets[n-1]
+
+

Graphically explained

+
    +
  1. Distribute elements in buckets:
    2. +
+

distribution step

+
    +
  1. Sorting inside every bucket and merging:
    2. +
+

sorting each bucket and merge

+

An example

+

Input

+

Suppose we have the following list of elements: [2, 56, 4, 77, 26, 98, 55]. Let's use 10 buckets. To determine the capacity of each bucket we need to know the maximum element value, in this case 98.

+

So the buckets are:

+ +

Distribution

+

Now we need to choose a distribution function.

+

bucketNumber = (elementValue / totalNumberOfBuckets) + 1

+

Such that by applying that function we distribute all the elements in the buckets.

+

In our example it is like the following:

+
    +
  1. Apply the distribution function to 2. bucketNumber = (2 / 10) + 1 = 1
    2. +
  2. Apply the distribution function to 56. bucketNumber = (56 / 10) + 1 = 6
    3. +
  3. Apply the distribution function to 4. bucketNumber = (4 / 10) + 1 = 1
    4. +
  4. Apply the distribution function to 77. bucketNumber = (77 / 10) + 1 = 8
    5. +
  5. Apply the distribution function to 26. bucketNumber = (26 / 10) + 1 = 3
    6. +
  6. Apply the distribution function to 98. bucketNumber = (98 / 10) + 1 = 10
    7. +
  7. Apply the distribution function to 55. bucketNumber = (55 / 10) + 1 = 6
    8. +
+

Our buckets will be filled now:

+

1 : [2, 4]
+2 : []
+3 : [26]
+4 : []
+5 : []
+6 : [55, 56]
+7 : []
+8 : [77]
+9 : []
+10 : [98]

+

We can choose to insert the elements in every bucket in order, or sort every bucket after distributing all the elements.

+

Put the elements back in the list

+

Finally we go through all the buckets and put the elements back in the list:

+

[2, 4, 26, 55, 56, 77, 98]

+

Swift implementation

+

Here is a diagram that shows the functions, data structures and protocols for our bucker sort implementation:

+

classes

+

Main function

+

bucketSort() is a generic function that can apply the algorithm to any element of type T, as long as T is Sortable.

+
public func bucketSort<T:Sortable>(elements: [T],
+                                distributor: Distributor,
+                                     sorter: Sorter,
+                                    buckets: [Bucket<T>]) -> [T] {
+	precondition(allPositiveNumbers(elements))
+	precondition(enoughSpaceInBuckets(buckets, elements: elements))
+
+	var bucketsCopy = buckets
+	for elem in elements {
+		distributor.distribute(elem, buckets: &bucketsCopy)
+	}
+
+	var results = [T]()
+
+	for bucket in bucketsCopy {
+		results += bucket.sort(sorter)
+	}
+
+	return results
+}
+

Bucket

+
public struct Bucket<T:Sortable> {
+	var elements: [T]
+	let capacity: Int
+
+	public init(capacity: Int) {
+		self.capacity = capacity
+		elements = [T]()
+	}
+
+	public mutating func add(item: T) {
+		if (elements.count < capacity) {
+			elements.append(item)
+		}
+	}
+
+	public func sort(algorithm: Sorter) -> [T] {
+		return algorithm.sort(elements)
+	}
+}
+

Sortable

+
public protocol Sortable: IntConvertible, Comparable {
+}
+

IntConvertible

+

The algorithm is designed to sort integers, so all the elements to be sorted should be mapped to an integer value.

+
public protocol IntConvertible {
+	func toInt() -> Int
+}
+

Sorter

+
public protocol Sorter {
+	func sort<T:Sortable>(items: [T]) -> [T]
+}
+

Distributor

+
public protocol Distributor {
+	func distribute<T:Sortable>(element: T, inout buckets: [Bucket<T>])
+}
+

Custom Sorter and Distributor

+

The current implementation make use of the following implementations for Sorter and Distributor.

+

Sorter

+
public struct InsertionSorter: Sorter {
+	public func sort<T:Sortable>(items: [T]) -> [T] {
+		var results = items
+		for i in 0 ..< results.count {
+			var j = i
+			while ( j > 0 && results[j-1] > results[j]) {
+
+				let auxiliar = results[j-1]
+				results[j-1] = results[j]
+				results[j] = auxiliar
+
+				j -= 1
+			}
+		}
+		return results
+	}
+}
+

Distributor

+
/*
+ * An example of a simple distribution function that send every elements to
+ * the bucket representing the range in which it fits.
+ *
+ * If the range of values to sort is 0..<49 i.e, there could be 5 buckets of capacity = 10
+ * So every element will be classified by the ranges:
+ *
+ * -  0 ..< 10
+ * - 10 ..< 20
+ * - 20 ..< 30
+ * - 30 ..< 40
+ * - 40 ..< 50
+ *
+ * By following the formula: element / capacity = #ofBucket
+ */
+public struct RangeDistributor: Distributor {
+	 public func distribute<T:Sortable>(element: T, inout buckets: [Bucket<T>]) {
+	 let value = element.toInt()
+	 let bucketCapacity = buckets.first!.capacity
+
+	 let bucketIndex = value / bucketCapacity
+	 buckets[bucketIndex].add(element)
+	}
+}
+

Make your own version

+

By reusing this code and implementing your own Sorter and Distributor you can experiment with different versions.

+

Other variations of Bucket Sort

+

The following are some of the variation to the general Bucket Sort implemented here:

+ +

Written for Swift Algorithm Club by Barbara Rodeker. Images from Wikipedia. Updated by Bruno Scheele.

+ + diff --git a/Closest Pair/index.html b/Closest Pair/index.html new file mode 100644 index 000000000..7a4ceff64 --- /dev/null +++ b/Closest Pair/index.html @@ -0,0 +1,86 @@ + + + Closest Pair + + + +

ClosestPair

+

Closest Pair is an algorithm that finds the closest pair of a given array of points By utilizing the Divide and Conquer methodology of solving problems so that it reaches the correct solution with O(nlogn) complexity.

+

Given points and we're required to find the two red ones

+

As we see in the above image there are an array of points and we need to find the closest two, But how do we do that without having to compare each two points which results in a whopping O(n^2) complexity?

+

Here is the main algorithm (Steps) we'll follow.

+ +
var innerPoints = mergeSort(points, sortAccording : true)
+ +
let line:Double = (p[mid].x + p[mid+1].x)/2
+

and just recursively calls itself until it reaches the base case we don't detect those points.

+

Points lying near the division line

+ +
var strip = [Point]()   
+var i=0, j = 0
+while i<n
+{
+	if abs(p[i].x - line) < min
+	{
+		strip.append(p[i])
+		j+=1
+	}
+	i+=1
+}
+ +

The strip with 4 points shown

+ +
while i<j
+    {
+        x = i+1
+        while x < j
+        {
+            if (abs(strip[x].y - strip[i].y)) > min { break }
+            if dist(strip[i], strip[x]) < temp
+            {
+                temp = dist(strip[i], strip[x])
+                tempFirst = strip[i]
+                tempSecond = strip[x]
+            }
+            x+=1
+        }
+        i+=1
+    }
+ +

So this is the rundown of how the algorithm works and you could see the fun little math tricks used to optimize this and we end up with O(nlogn) complexity mainly because of the sorting.

+

See also

+

See the playground to play around with the implementation of the algorithm

+

Wikipedia

+

Written for Swift Algorithm Club by Ahmed Nader

+ + diff --git a/Comb Sort/index.html b/Comb Sort/index.html new file mode 100644 index 000000000..e0234eb54 --- /dev/null +++ b/Comb Sort/index.html @@ -0,0 +1,61 @@ + + + Comb Sort + + + +

Comb Sort

+

A common issue for Bubble Sort is when small values are located near the end of an array.
+This problem severely slows down Bubble Sort, as it must move the small value -- or turtle --
+through nearly the entire array. Bubble Sort works by checking the current index of an array
+against the next index, and when those two values are unsorted, they are swapped into place.
+As a result, the values bubble into their rightful place within the array.

+

Comb Sort improves upon Bubble Sort by dealing with these turtles near the end of the array.
+The value of the current index of the array is compared against one a set distance away. This
+removes a worst-case scenario of Bubble Sort, and greatly improves on the time complexity of Bubble Sort.

+

Example

+

A step-by-step example of how Comb Sort works, and differs from Bubble Sort, can be seen here.

+

Here is a visual to see Comb Sort in effect:

+

+

Algorithm

+

Similar to Bubble Sort, two values within an array are compared. When the lower index value
+is larger than the higher index value, and thus out of place within the array, they are
+swapped. Unlike Bubble Sort, the value being compared against is a set distance away. This
+value -- the gap -- is slowly decreased through iterations.

+

The Code

+

Here is a Swift implementation of Comb Sort:

+
func combSort (input: [Int]) -> [Int] {
+    var copy: [Int] = input
+    var gap = copy.count
+    let shrink = 1.3
+
+    while gap > 1 {
+        gap = (Int)(Double(gap) / shrink)
+        if gap < 1 {
+            gap = 1
+        }
+    
+        var index = 0
+        while !(index + gap >= copy.count) {
+            if copy[index] > copy[index + gap] {
+                swap(&copy[index], &copy[index + gap])
+            }
+            index += 1
+        }
+    }
+    return copy
+}
+

This code can be tested in a playground by calling this method with a paramaterized array to sort:

+
combSort(example_array_of_values)
+

This will sort the values of the array into ascending order -- increasing in value.

+

Performance

+

Comb Sort was created to improve upon the worst case time complexity of Bubble Sort. With Comb
+Sort, the worst case scenario for performance is polynomial -- O(n^2). At best though, Comb Sort
+performs at O(n logn) time complexity -- loglinear. This creates a drastic improvement over Bubble Sort's performance.

+

Similar to Bubble Sort, the space complexity for Comb Sort is constant -- O(1).
+This is extremely space efficient as it sorts the array in place.

+

Additional Resources

+

Comb Sort Wikipedia

+

Written for the Swift Algorithm Club by Stephen Rutstein

+ + diff --git a/Combinatorics/index.html b/Combinatorics/index.html new file mode 100644 index 000000000..f1bd9ba34 --- /dev/null +++ b/Combinatorics/index.html @@ -0,0 +1,276 @@ + + + Combinatorics + + + +

Permutations

+

A permutation is a certain arrangement of the objects from a collection. For example, if we have the first five letters from the alphabet, then this is a permutation:

+
a, b, c, d, e
+
+

This is another permutation:

+
b, e, d, a, c
+
+

For a collection of n objects, there are n! possible permutations, where ! is the "factorial" function. So for our collection of five letters, the total number of permutations you can make is:

+
5! = 5 * 4 * 3 * 2 * 1 = 120
+
+

A collection of six items has 6! = 720 permutations. For ten items, it is 10! = 3,628,800. That adds up quick!

+

Where does this n! come from? The logic is as follows: we have a collection of five letters that we want to put in some order. To do this, you need to pick up these letters one-by-one. Initially, you have the choice of five letters: a, b, c, d, e. That gives 5 possibilities.

+

After picking the first letter, you only have four letters left to choose from. That gives 5 * 4 = 20 possibilities:

+
a+b    b+a    c+a    d+a    e+a
+a+c    b+c    c+b    d+b    e+b
+a+d    b+d    c+d    d+c    e+c
+a+e    b+e    c+e    d+e    e+d
+
+

After picking the second letter, there are only three letters left to choose from. And so on... When you get to the last letter, you don't have any choice because there is only one letter left. That's why the total number of possibilities is 5 * 4 * 3 * 2 * 1.

+

To calculate the factorial in Swift:

+
func factorial(_ n: Int) -> Int {
+  var n = n
+  var result = 1
+  while n > 1 {
+    result *= n
+    n -= 1
+  }
+  return result
+}
+

Try it out in a playground:

+
factorial(5)   // returns 120
+

Note that factorial(20) is the largest number you can calculate with this function, or you'll get integer overflow.

+

Let's say that from that collection of five letters you want to choose only 3 elements. How many possible ways can you do this? Well, that works the same way as before, except that you stop after the third letter. So now the number of possibilities is 5 * 4 * 3 = 60.

+

The formula for this is:

+
             n!
+P(n, k) = --------
+          (n - k)!
+
+

where n is the size of your collection and k is the size of the group that you're selecting. In our example, P(5, 3) = 5! / (5 - 3)! = 120 / 2 = 60.

+

You could implement this in terms of the factorial() function from earlier, but there's a problem. Remember that factorial(20) is the largest possible number it can handle, so you could never calculate P(21, 3), for example.

+

Here is an algorithm that can deal with larger numbers:

+
func permutations(_ n: Int, _ k: Int) -> Int {
+  var n = n
+  var answer = n
+  for _ in 1..<k {
+    n -= 1
+    answer *= n
+  }
+  return answer
+}
+

Try it out:

+
permutations(5, 3)   // returns 60
+permutations(50, 6)  // returns 11441304000
+permutations(9, 4)   // returns 3024
+

This function takes advantage of the following algebra fact:

+
          9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1
+P(9, 4) = --------------------------------- = 9 * 8 * 7 * 6 = 3024
+                          5 * 4 * 3 * 2 * 1
+
+

The denominator cancels out part of the numerator, so there's no need to perform a division and you're not dealing with intermediate results that are potentially too large.

+

However, there are still limits to what you can calculate; for example the number of groups of size 15 that you can make from a collection of 30 objects -- i.e. P(30, 15) -- is ginormous and breaks Swift. Huh, you wouldn't think it would be so large but combinatorics is funny that way.

+

Generating the permutations

+

So far we've counted how many permutations exist for a given collection, but how can we actually create a list of all these permutations?

+

Here's a recursive algorithm by Niklaus Wirth:

+
func permuteWirth<T>(_ a: [T], _ n: Int) {
+    if n == 0 {
+        print(a)   // display the current permutation
+    } else {
+        var a = a
+        permuteWirth(a, n - 1)
+        for i in 0..<n {
+            a.swapAt(i, n)
+            permuteWirth(a, n - 1)
+            a.swapAt(i, n)
+        }
+    }
+}
+

Use it as follows:

+
let letters = ["a", "b", "c", "d", "e"]
+permuteWirth(letters, letters.count - 1)
+

This prints all the permutations of the input array to the debug output:

+
["a", "b", "c", "d", "e"]
+["b", "a", "c", "d", "e"]
+["c", "b", "a", "d", "e"]
+["b", "c", "a", "d", "e"]
+["a", "c", "b", "d", "e"]
+...
+

As we've seen before, there will be 120 of them.

+

How does the algorithm work? Good question! Let's step through a simple example with just three elements. The input array is:

+
[ "x", "y", "z" ]
+
+

We're calling it like so:

+
permuteWirth([ "x", "y", "z" ], 2)
+

Note that the n parameter is one less than the number of elements in the array!

+

After calling permuteWirth() it immediately calls itself recursively with n = 1. And that immediately calls itself recursively again with n = 0. The call tree looks like this:

+
permuteWirth([ "x", "y", "z" ], 2)
+	permuteWirth([ "x", "y", "z" ], 1)
+		permuteWirth([ "x", "y", "z" ], 0)   // prints ["x", "y", "z"]
+

When n is equal to 0, we print out the current array, which is still unchanged at this point. The recursion has reached the base case, so now we go back up one level and enter the for loop.

+
permuteWirth([ "x", "y", "z" ], 2)
+	permuteWirth([ "x", "y", "z" ], 1)   <--- back to this level
+	    swap a[0] with a[1]
+        permuteWirth([ "y", "x", "z" ], 0)   // prints ["y", "x", "z"]
+	    swap a[0] and a[1] back
+

This swapped "y" and "x" and printed the result. We're done at this level of the recursion and go back to the top. This time we do two iterations of the for loop because n = 2 here. The first iteration looks like this:

+
permuteWirth([ "x", "y", "z" ], 2)   <--- back to this level
+    swap a[0] with a[2]
+	permuteWirth([ "z", "y", "x" ], 1)
+        permuteWirth([ "z", "y", "x" ], 0)   // prints ["z", "y", "x"]
+	    swap a[0] with a[1]
+        permuteWirth([ "y", "z", "x" ], 0)   // prints ["y", "z", "x"]
+	    swap a[0] and a[1] back
+    swap a[0] and a[2] back
+

And the second iteration:

+
permuteWirth([ "x", "y", "z" ], 2)
+    swap a[1] with a[2]                 <--- second iteration of the loop
+	permuteWirth([ "x", "z", "y" ], 1)
+        permuteWirth([ "x", "z", "y" ], 0)   // prints ["x", "z", "y"]
+	    swap a[0] with a[1]
+        permuteWirth([ "z", "x", "y" ], 0)   // prints ["z", "x", "y"]
+	    swap a[0] and a[1] back
+    swap a[1] and a[2] back
+

To summarize, first it swaps these items:

+
[ 2, 1, - ]
+
+

Then it swaps these:

+
[ 3, -, 1 ]
+
+

Recursively, it swaps the first two again:

+
[ 2, 3, - ]
+
+

Then it goes back up one step and swaps these:

+
[ -, 3, 2 ]
+
+

And finally the first two again:

+
[ 3, 1, - ]
+
+

Of course, the larger your array is, the more swaps it performs and the deeper the recursion gets.

+

If the above is still not entirely clear, then I suggest you give it a go in the playground. That's what playgrounds are great for. :-)

+

For fun, here is an alternative algorithm, by Robert Sedgewick:

+
func permuteSedgewick(_ a: [Int], _ n: Int, _ pos: inout Int) {
+  var a = a
+  pos += 1
+  a[n] = pos
+  if pos == a.count - 1 {
+    print(a)              // display the current permutation
+  } else {
+    for i in 0..<a.count {
+      if a[i] == 0 {
+        permuteSedgewick(a, i, &pos)
+      }
+    }
+  }
+  pos -= 1
+  a[n] = 0
+}
+

You use it like this:

+
let numbers = [0, 0, 0, 0]
+var pos = -1
+permuteSedgewick(numbers, 0, &pos)
+

The array must initially contain all zeros. 0 is used as a flag that indicates more work needs to be done on each level of the recursion.

+

The output of the Sedgewick algorithm is:

+
[1, 2, 3, 0]
+[1, 2, 0, 3]
+[1, 3, 2, 0]
+[1, 0, 2, 3]
+[1, 3, 0, 2]
+...
+

It can only deal with numbers, but these can serve as indices into the actual array you're trying to permute, so it's just as powerful as Wirth's algorithm.

+

Try to figure out for yourself how this algorithm works!

+

Combinations

+

A combination is like a permutation where the order does not matter. The following are six different permutations of the letters k l m but they all count as the same combination:

+
k, l, m      k, m, l      m, l, k
+l, m, k      l, k, m      m, k, l
+
+

So there is only one combination of size 3. However, if we're looking for combinations of size 2, we can make three:

+
k, l      (is the same as l, k)
+l, m      (is the same as m, l)
+k, m      (is the same as m, k)
+
+

The C(n, k) function counts the number of ways to choose k things out of n possibilities. That's why it's also called "n-choose-k". (A fancy mathematical term for this number is "binomial coefficient".)

+

The formula for C(n, k) is:

+
               n!         P(n, k)
+C(n, k) = ------------- = --------
+          (n - k)! * k!      k!
+
+

As you can see, you can derive it from the formula for P(n, k). There are always more permutations than combinations. You divide the number of permutations by k! because a total of k! of these permutations give the same combination.

+

Above I showed that the number of permutations of k l m is 6, but if you pick only two of those letters the number of combinations is 3. If we use the formula we should get the same answer. We want to calculate C(3, 2) because we choose 2 letters out of a collection of 3.

+
          3 * 2 * 1    6
+C(3, 2) = --------- = --- = 3
+           1! * 2!     2
+
+

Here's a simple function to calculate C(n, k):

+
func combinations(_ n: Int, choose k: Int) -> Int {
+  return permutations(n, k) / factorial(k)
+}
+

Use it like this:

+
combinations(28, choose: 5)    // prints 98280
+

Because this uses the permutations() and factorial() functions under the hood, you're still limited by how large these numbers can get. For example, combinations(30, 15) is "only" 155,117,520 but because the intermediate results don't fit into a 64-bit integer, you can't calculate it with the given function.

+

There's a faster approach to calculate C(n, k) in O(k) time and O(1) extra space. The idea behind it is that the formula for C(n, k) is:

+
               n!                      n * (n - 1) * ... * 1
+C(n, k) = ------------- = ------------------------------------------
+          (n - k)! * k!      (n - k) * (n - k - 1) * ... * 1 * k!
+
+

After the reduction of fractions, we get the following formula:

+
               n * (n - 1) * ... * (n - k + 1)         (n - 0) * (n - 1) * ... * (n - k + 1)
+C(n, k) = --------------------------------------- = -----------------------------------------
+                           k!                          (0 + 1) * (1 + 1) * ... * (k - 1 + 1)
+
+

We can implement this formula as follows:

+
func quickBinomialCoefficient(_ n: Int, choose k: Int) -> Int {
+  var result = 1
+  for i in 0..<k {
+    result *= (n - i)
+    result /= (i + 1)
+  }
+  return result
+}
+

This algorithm can create larger numbers than the previous method. Instead of calculating the entire numerator (a potentially huge number) and then dividing it by the factorial (also a very large number), here we already divide in each step. That causes the temporary results to grow much less quickly.

+

Here's how you can use this improved algorithm:

+
quickBinomialCoefficient(8, choose: 2)     // prints 28
+quickBinomialCoefficient(30, choose: 15)   // prints 155117520
+

This new method is quite fast but you're still limited in how large the numbers can get. You can calculate C(30, 15) without any problems, but something like C(66, 33) will still cause integer overflow in the numerator.

+

Here is an algorithm that uses dynamic programming to overcome the need for calculating factorials and doing divisions. It is based on Pascal's triangle:

+
0:               1
+1:             1   1
+2:           1   2   1
+3:         1   3   3   1
+4:       1   4   6   4   1
+5:     1   5  10   10  5   1
+6:   1   6  15  20   15  6   1
+
+

Each number in the next row is made up by adding two numbers from the previous row. For example in row 6, the number 15 is made by adding the 5 and 10 from row 5. These numbers are called the binomial coefficients and as it happens they are the same as C(n, k).

+

For example, for row 6:

+
C(6, 0) = 1
+C(6, 1) = 6
+C(6, 2) = 15
+C(6, 3) = 20
+C(6, 4) = 15
+C(6, 5) = 6
+C(6, 6) = 1
+
+

The following code calculates Pascal's triangle in order to find the C(n, k) you're looking for:

+
func binomialCoefficient(_ n: Int, choose k: Int) -> Int {
+  var bc = Array(repeating: Array(repeating: 0, count: n + 1), count: n + 1)
+
+  for i in 0...n {
+    bc[i][0] = 1
+    bc[i][i] = 1
+  }
+
+  if n > 0 {
+    for i in 1...n {
+      for j in 1..<i {
+        bc[i][j] = bc[i - 1][j - 1] + bc[i - 1][j]
+      }
+    }
+  }
+
+  return bc[n][k]
+}
+

The algorithm itself is quite simple: the first loop fills in the 1s at the outer edges of the triangle. The other loops calculate each number in the triangle by adding up the two numbers from the previous row.

+

Now you can calculate C(66, 33) without any problems:

+
binomialCoefficient(66, choose: 33)   // prints a very large number
+

You may wonder what the point is in calculating these permutations and combinations, but many algorithm problems are really combinatorics problems in disguise. Often you may need to look at all possible combinations of your data to see which one gives the right solution. If that means you need to search through n! potential solutions, you may want to consider a different approach -- as you've seen, these numbers become huge very quickly!

+

References

+

Wirth's and Sedgewick's permutation algorithms and the code for counting permutations and combinations are based on the Algorithm Alley column from Dr.Dobb's Magazine, June 1993. The dynamic programming binomial coefficient algorithm is from The Algorithm Design Manual by Skiena.

+

Written for Swift Algorithm Club by Matthijs Hollemans and Kanstantsin Linou

+ + diff --git a/Convex Hull/index.html b/Convex Hull/index.html new file mode 100644 index 000000000..97d67f78f --- /dev/null +++ b/Convex Hull/index.html @@ -0,0 +1,46 @@ + + + Convex Hull + + + +

Convex Hull

+

Given a group of points on a plane. The Convex Hull algorithm calculates the shape (made up from the points itself) containing all these points. It can also be used on a collection of points of different dimensions. This implementation however covers points on a plane. It essentially calculates the lines between points which together contain all points. In comparing different solutions to this problem we can describe each algorithm in terms of it's big-O time complexity.

+

There are multiple Convex Hull algorithms but this solution is called Quickhull, is comes from the work of both W. Eddy in 1977 and also separately A. Bykat in 1978, this algorithm has an expected time complexity of O(n log n), but it's worst-case time-complexity can be O(n^2) . With average conditions the algorithm has ok efficiency, but it's time-complexity can start to become more exponential in cases of high symmetry or where there are points lying on the circumference of a circle for example.

+

Quickhull

+

The quickhull algorithm works as follows:

+ +

Our function will have the following defininition:

+

findHull(points: [CGPoint], p1: CGPoint, p2: CGPoint)

+
findHull(S1, A, B)
+findHull(S2, B, A)
+
+

What this function does is the following:

+
    +
  1. If points is empty we return as there are no points to the right of our line to add to our hull.
    2. +
  2. Draw a line from p1 to p2.
    3. +
  3. Find the point in points that is furthest away from this line. (maxPoint)
    4. +
  4. Add maxPoint to the hull right after p1.
    5. +
  5. Draw a line (line1) from p1 to maxPoint.
    6. +
  6. Draw a line (line2) from maxPoint to p2. (These lines now form a triangle)
    7. +
  7. All points within this triangle are of course not part of the hull and thus can be ignored. We check which points in points are to the right of line1 these are grouped in an array s1.
    8. +
  8. All points that are to the right of line2 are grouped in an array s2. Note that there are no points that are both to the right of line1 and line2 as then maxPoint wouldn't be the point furthest away from our initial line between p1 and p2.
    9. +
  9. We call findHull(_, _, _) again on our new groups of points to find more hull points.
    10. +
+
findHull(s1, p1, maxPoint)
+findHull(s2, maxPoint, p2)
+
+

This eventually leaves us with an array of points describing the convex hull.

+

See also

+

Convex Hull on Wikipedia

+

Written for the Swift Algorithm Club by Jaap Wijnen.

+ + diff --git a/Count Occurrences/index.html b/Count Occurrences/index.html new file mode 100644 index 000000000..995974b56 --- /dev/null +++ b/Count Occurrences/index.html @@ -0,0 +1,100 @@ + + + Count Occurrences + + + +

Count Occurrences

+

Goal: Count how often a certain value appears in an array.

+

The obvious way to do this is with a linear search from the beginning of the array until the end, keeping count of how often you come across the value. That is an O(n) algorithm.

+

However, if the array is sorted you can do it much faster, in O(log n) time, by using a modification of binary search.

+

Let's say we have the following array:

+
[ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+
+

If we want to know how often the value 3 occurs, we can do a regular binary search for 3. That could give us any of these four indices:

+
[ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+           *  *  *  *
+
+

But that still doesn't tell you how many other 3s there are. To find those other 3s, you'd still have to do a linear search to the left and a linear search to the right. That will be fast enough in most cases, but in the worst case -- when the array consists of nothing but 3s -- it still takes O(n) time.

+

The trick is to use two binary searches, one to find where the 3s start (the left boundary), and one to find where they end (the right boundary).

+

In code this looks as follows:

+
func countOccurrences<T: Comparable>(of key: T, in array: [T]) -> Int {
+  var leftBoundary: Int {
+    var low = 0
+    var high = a.count
+    while low < high {
+      let midIndex = low + (high - low)/2
+      if a[midIndex] < key {
+        low = midIndex + 1
+      } else {
+        high = midIndex
+      }
+    }
+    return low
+  }
+
+  var rightBoundary: Int {
+    var low = 0
+    var high = a.count
+    while low < high {
+      let midIndex = low + (high - low)/2
+      if a[midIndex] > key {
+        high = midIndex
+      } else {
+        low = midIndex + 1
+      }
+    }
+    return low
+  }
+
+  return rightBoundary - leftBoundary
+}
+

Notice that the variables leftBoundary and rightBoundary are very similar to the binary search algorithm. The big difference is that they don't stop when they find the search key, but keep going. Also, notice that we constrain the type T to be Comparable so that the algorithm can be applied to an array of Strings, Ints or other types that conform to the Swift Comparable protocol.

+

To test this algorithm, copy the code to a playground and then do:

+
let a = [ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+
+countOccurrencesOfKey(3, inArray: a)  // returns 4
+
+

Remember: If you use your own array, make sure it is sorted first!

+
+

Let's walk through the example. The array is:

+
[ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+
+

To find the left boundary, we start with low = 0 and high = 12. The first mid index is 6:

+
[ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+                    *
+
+

With a regular binary search you'd be done now, but here we're not just looking whether the value 3 occurs or not -- instead, we want to find where it occurs first.

+

Since this algorithm follows the same principle as binary search, we now ignore the right half of the array and calculate the new mid index:

+
[ 0, 1, 1, 3, 3, 3 | x, x, x, x, x, x ]
+           *
+
+

Again, we've landed on a 3, and it's the very first one. But the algorithm doesn't know that, so we split the array again:

+
[ 0, 1, 1 | x, x, x | x, x, x, x, x, x ]
+     *
+
+

Still not done. Split again, but this time use the right half:

+
[ x, x | 1 | x, x, x | x, x, x, x, x, x ]
+         *
+
+

The array cannot be split up any further, which means we've found the left boundary, at index 3.

+

Now let's start over and try to find the right boundary. This is very similar, so I'll just show you the different steps:

+
[ 0, 1, 1, 3, 3, 3, 3, 6, 8, 10, 11, 11 ]
+                    *
+
+[ x, x, x, x, x, x, x | 6, 8, 10, 11, 11 ]
+                              *
+
+[ x, x, x, x, x, x, x | 6, 8, | x, x, x ]
+                           *
+
+[ x, x, x, x, x, x, x | 6 | x | x, x, x ]
+                        *
+
+

The right boundary is at index 7. The difference between the two boundaries is 7 - 3 = 4, so the number 3 occurs four times in this array.

+

Each binary search took 4 steps, so in total this algorithm took 8 steps. Not a big gain on an array of only 12 items, but the bigger the array, the more efficient this algorithm becomes. For a sorted array with 1,000,000 items, it only takes 2 x 20 = 40 steps to count the number of occurrences for any particular value.

+

By the way, if the value you're looking for is not in the array, then rightBoundary and leftBoundary return the same value and so the difference between them is 0.

+

This is an example of how you can modify the basic binary search to solve other algorithmic problems as well. Of course, it does require that the array is sorted.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Counting Sort/index.html b/Counting Sort/index.html new file mode 100644 index 000000000..23ebfed31 --- /dev/null +++ b/Counting Sort/index.html @@ -0,0 +1,53 @@ + + + Counting Sort + + + +

Counting Sort

+

Counting sort is an algorithm for sorting a collection of objects according to keys that are small integers. It operates by counting the number of objects that have each distinct key values, and using arithmetic on those counts to determine the positions of each key value in the output sequence.

+

Example

+

To understand the algorithm let's walk through a small example.

+

Consider the array: [ 10, 9, 8, 7, 1, 2, 7, 3 ]

+

Step 1:

+

The first step is to count the total number of occurrences for each item in the array. The output for the first step would be a new array that looks as follows:

+
Index 0 1 2 3 4 5 6 7 8 9 10
+Count 0 1 1 1 0 0 0 2 1 1 1
+
+

Here is the code to accomplish this:

+
  let maxElement = array.max() ?? 0
+
+  var countArray = [Int](repeating: 0, count: Int(maxElement + 1))
+  for element in array {
+    countArray[element] += 1
+  }
+

Step 2:

+

In this step the algorithm tries to determine the number of elements that are placed before each element. Since, you already know the total occurrences for each element you can use this information to your advantage. The way it works is to sum up the previous counts and store them at each index.

+

The count array would be as follows:

+
Index 0 1 2 3 4 5 6 7 8 9 10
+Count 0 1 2 3 3 3 3 5 6 7 8
+
+

The code for step 2 is:

+
  for index in 1 ..< countArray.count {
+    let sum = countArray[index] + countArray[index - 1]
+    countArray[index] = sum
+  }
+

Step 3:

+

This is the last step in the algorithm. Each element in the original array is placed at the position defined by the output of step 2. For example, the number 10 would be placed at an index of 7 in the output array. Also, as you place the elements you need to reduce the count by 1 as those many elements are reduced from the array.

+

The final output would be:

+
Index  0 1 2 3 4 5 6 7
+Output 1 2 3 7 7 8 9 10
+
+

Here is the code for this final step:

+
  var sortedArray = [Int](repeating: 0, count: array.count)
+  for element in array {
+    countArray[element] -= 1
+    sortedArray[countArray[element]] = element
+  }
+  return sortedArray
+

Performance

+

The algorithm uses simple loops to sort a collection. Hence, the time to run the entire algorithm is O(n+k) where O(n) represents the loops that are required to initialize the output arrays and O(k) is the loop required to create the count array.

+

The algorithm uses arrays of length n + 1 and n, so the total space required is O(2n). Hence for collections where the keys are scattered in a dense area along the number line it can be space efficient.

+

Written for Swift Algorithm Club by Ali Hafizji

+ + diff --git a/Depth-First Search/index.html b/Depth-First Search/index.html new file mode 100644 index 000000000..5c78497d7 --- /dev/null +++ b/Depth-First Search/index.html @@ -0,0 +1,71 @@ + + + Depth-First Search + + + +

Depth-First Search

+
+

This topic has been tutorialized here

+
+

Depth-first search (DFS) is an algorithm for traversing or searching tree or graph data structures. It starts at a source node and explores as far as possible along each branch before backtracking.

+

Depth-first search can be used on both directed and undirected graphs.

+

Animated example

+

Here's how depth-first search works on a graph:

+

Animated example

+

Let's say we start the search from node A. In depth-first search we look at the starting node's first neighbor and visit that. In the example that is node B. Then we look at node B's first neighbor and visit it. This is node D. Since D doesn't have any unvisited neighbors of its own, we backtrack to node B and go to its other neighbor E. And so on, until we've visited all the nodes in the graph.

+

Each time we visit the first neighbor and keep going until there's nowhere left to go, and then we backtrack to a point where there are again nodes to visit. When we've backtracked all the way to node A, the search is complete.

+

For the example, the nodes were visited in the order A, B, D, E, H, F, G, C.

+

The depth-first search process can also be visualized as a tree:

+

Traversal tree

+

The parent of a node is the one that "discovered" that node. The root of the tree is the node you started the depth-first search from. Whenever there's a branch, that's where we backtracked.

+

The code

+

Simple recursive implementation of depth-first search:

+
func depthFirstSearch(_ graph: Graph, source: Node) -> [String] {
+  var nodesExplored = [source.label]
+  source.visited = true
+
+  for edge in source.neighbors {
+    if !edge.neighbor.visited {
+      nodesExplored += depthFirstSearch(graph, source: edge.neighbor)
+    }
+  }
+  return nodesExplored
+}
+

Where a breadth-first search visits all immediate neighbors first, a depth-first search tries to go as deep down the tree or graph as it can.

+

Put this code in a playground and test it like so:

+
let graph = Graph()
+
+let nodeA = graph.addNode("a")
+let nodeB = graph.addNode("b")
+let nodeC = graph.addNode("c")
+let nodeD = graph.addNode("d")
+let nodeE = graph.addNode("e")
+let nodeF = graph.addNode("f")
+let nodeG = graph.addNode("g")
+let nodeH = graph.addNode("h")
+
+graph.addEdge(nodeA, neighbor: nodeB)
+graph.addEdge(nodeA, neighbor: nodeC)
+graph.addEdge(nodeB, neighbor: nodeD)
+graph.addEdge(nodeB, neighbor: nodeE)
+graph.addEdge(nodeC, neighbor: nodeF)
+graph.addEdge(nodeC, neighbor: nodeG)
+graph.addEdge(nodeE, neighbor: nodeH)
+graph.addEdge(nodeE, neighbor: nodeF)
+graph.addEdge(nodeF, neighbor: nodeG)
+
+let nodesExplored = depthFirstSearch(graph, source: nodeA)
+print(nodesExplored)
+

This will output: ["a", "b", "d", "e", "h", "f", "g", "c"]

+

What is DFS good for?

+

Depth-first search can be used to solve many problems, for example:

+ +

Written for Swift Algorithm Club by Paulo Tanaka and Matthijs Hollemans

+ + diff --git a/Deque/index.html b/Deque/index.html new file mode 100644 index 000000000..83cd1432a --- /dev/null +++ b/Deque/index.html @@ -0,0 +1,257 @@ + + + Deque + + + +

Deque

+

A double-ended queue. For some reason this is pronounced as "deck".

+

A regular queue adds elements to the back and removes from the front. The deque also allows enqueuing at the front and dequeuing from the back, and peeking at both ends.

+

Here is a very basic implementation of a deque in Swift:

+
public struct Deque<T> {
+  private var array = [T]()
+
+  public var isEmpty: Bool {
+    return array.isEmpty
+  }
+
+  public var count: Int {
+    return array.count
+  }
+
+  public mutating func enqueue(_ element: T) {
+    array.append(element)
+  }
+
+  public mutating func enqueueFront(_ element: T) {
+    array.insert(element, at: 0)
+  }
+
+  public mutating func dequeue() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array.removeFirst()
+    }
+  }
+
+  public mutating func dequeueBack() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array.removeLast()
+    }
+  }
+
+  public func peekFront() -> T? {
+    return array.first
+  }
+
+  public func peekBack() -> T? {
+    return array.last
+  }
+}
+

This uses an array internally. Enqueuing and dequeuing are simply a matter of adding and removing items from the front or back of the array.

+

An example of how to use it in a playground:

+
var deque = Deque<Int>()
+deque.enqueue(1)
+deque.enqueue(2)
+deque.enqueue(3)
+deque.enqueue(4)
+
+deque.dequeue()       // 1
+deque.dequeueBack()   // 4
+
+deque.enqueueFront(5)
+deque.dequeue()       // 5
+

This particular implementation of Deque is simple but not very efficient. Several operations are O(n), notably enqueueFront() and dequeue(). I've included it only to show the principle of what a deque does.

+

A more efficient version

+

The reason that dequeue() and enqueueFront() are O(n) is that they work on the front of the array. If you remove an element at the front of an array, what happens is that all the remaining elements need to be shifted in memory.

+

Let's say the deque's array contains the following items:

+
[ 1, 2, 3, 4 ]
+
+

Then dequeue() will remove 1 from the array and the elements 2, 3, and 4, are shifted one position to the front:

+
[ 2, 3, 4 ]
+
+

This is an O(n) operation because all array elements need to be moved by one position in the computer's memory.

+

Likewise, inserting an element at the front of the array is expensive because it requires that all other elements must be shifted one position to the back. So enqueueFront(5) will change the array to be:

+
[ 5, 2, 3, 4 ]
+
+

First, the elements 2, 3, and 4 are moved up by one position in the computer's memory, and then the new element 5 is inserted at the position where 2 used to be.

+

Why is this not an issue at for enqueue() and dequeueBack()? Well, these operations are performed at the end of the array. The way resizable arrays are implemented in Swift is by reserving a certain amount of free space at the back.

+

Our initial array [ 1, 2, 3, 4] actually looks like this in memory:

+
[ 1, 2, 3, 4, x, x, x ]
+
+

where the xs denote additional positions in the array that are not being used yet. Calling enqueue(6) simply copies the new item into the next unused spot:

+
[ 1, 2, 3, 4, 6, x, x ]
+
+

The dequeueBack() function uses array.removeLast() to delete that item. This does not shrink the array's memory but only decrements array.count by one. There are no expensive memory copies involved here. So operations at the back of the array are fast, O(1).

+

It is possible the array runs out of free spots at the back. In that case, Swift will allocate a new, larger array and copy over all the data. This is an O(n) operation but because it only happens once in a while, adding new elements at the end of an array is still O(1) on average.

+

Of course, we can use this same trick at the beginning of the array. That will make our deque efficient too for operations at the front of the queue. Our array will look like this:

+
[ x, x, x, 1, 2, 3, 4, x, x, x ]
+
+

There is now also a chunk of free space at the start of the array, which allows adding or removing elements at the front of the queue to be O(1) as well.

+

Here is the new version of Deque:

+
public struct Deque<T> {
+  private var array: [T?]
+  private var head: Int
+  private var capacity: Int
+  private let originalCapacity:Int
+
+  public init(_ capacity: Int = 10) {
+    self.capacity = max(capacity, 1)
+    originalCapacity = self.capacity
+    array = [T?](repeating: nil, count: capacity)
+    head = capacity
+  }
+
+  public var isEmpty: Bool {
+    return count == 0
+  }
+
+  public var count: Int {
+    return array.count - head
+  }
+
+  public mutating func enqueue(_ element: T) {
+    array.append(element)
+  }
+
+  public mutating func enqueueFront(_ element: T) {
+    // this is explained below
+  }
+
+  public mutating func dequeue() -> T? {
+    // this is explained below
+  }
+
+  public mutating func dequeueBack() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array.removeLast()
+    }
+  }
+
+  public func peekFront() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array[head]
+    }
+  }
+
+  public func peekBack() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array.last!
+    }
+  }  
+}
+

It still largely looks the same -- enqueue() and dequeueBack() haven't changed -- but there are also a few important differences. The array now stores objects of type T? instead of just T because we need some way to mark array elements as being empty.

+

The init method allocates a new array that contains a certain number of nil values. This is the free room we have to work with at the beginning of the array. By default this creates 10 empty spots.

+

The head variable is the index in the array of the front-most object. Since the queue is currently empty, head points at an index beyond the end of the array.

+
[ x, x, x, x, x, x, x, x, x, x ]
+                                 |
+                                 head
+
+

To enqueue an object at the front, we move head one position to the left and then copy the new object into the array at index head. For example, enqueueFront(5) gives:

+
[ x, x, x, x, x, x, x, x, x, 5 ]
+                             |
+                             head
+
+

Followed by enqueueFront(7):

+
[ x, x, x, x, x, x, x, x, 7, 5 ]
+                          |
+                          head
+
+

And so on... the head keeps moving to the left and always points at the first item in the queue. enqueueFront() is now O(1) because it only involves copying a value into the array, a constant-time operation.

+

Here is the code:

+
  public mutating func enqueueFront(element: T) {
+    head -= 1
+    array[head] = element
+  }
+

Appending to the back of the queue has not changed (it's the exact same code as before). For example, enqueue(1) gives:

+
[ x, x, x, x, x, x, x, x, 7, 5, 1, x, x, x, x, x, x, x, x, x ]
+                          |
+                          head
+
+

Notice how the array has resized itself. There was no room to add the 1, so Swift decided to make the array larger and add a number of empty spots to the end. If you enqueue another object, it gets added to the next empty spot in the back. For example, enqueue(2):

+
[ x, x, x, x, x, x, x, x, 7, 5, 1, 2, x, x, x, x, x, x, x, x ]
+                          |
+                          head
+
+
+

Note: You won't see those empty spots at the back of the array when you print(deque.array). This is because Swift hides them from you. Only the ones at the front of the array show up.

+
+

The dequeue() method does the opposite of enqueueFront(), it reads the value at head, sets the array element back to nil, and then moves head one position to the right:

+
  public mutating func dequeue() -> T? {
+    guard head < array.count, let element = array[head] else { return nil }
+
+    array[head] = nil
+    head += 1
+
+    return element
+  }
+

There is one tiny problem... If you enqueue a lot of objects at the front, you're going to run out of empty spots at the front at some point. When this happens at the back of the array, Swift automatically resizes it. But at the front of the array we have to handle this situation ourselves, with some extra logic in enqueueFront():

+
  public mutating func enqueueFront(element: T) {
+    if head == 0 {
+      capacity *= 2
+      let emptySpace = [T?](repeating: nil, count: capacity)
+      array.insert(contentsOf: emptySpace, at: 0)
+      head = capacity
+    }
+
+    head -= 1
+    array[head] = element
+  }
+

If head equals 0, there is no room left at the front. When that happens, we add a whole bunch of new nil elements to the array. This is an O(n) operation but since this cost gets divided over all the enqueueFront()s, each individual call to enqueueFront() is still O(1) on average.

+
+

Note: We also multiply the capacity by 2 each time this happens, so if your queue grows bigger and bigger, the resizing happens less often. This is also what Swift arrays automatically do at the back.

+
+

We have to do something similar for dequeue(). If you mostly enqueue a lot of elements at the back and mostly dequeue from the front, then you may end up with an array that looks as follows:

+
[ x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, 1, 2, 3 ]
+                                                              |
+                                                              head
+
+

Those empty spots at the front only get used when you call enqueueFront(). But if enqueuing objects at the front happens only rarely, this leaves a lot of wasted space. So let's add some code to dequeue() to clean this up:

+
  public mutating func dequeue() -> T? {
+    guard head < array.count, let element = array[head] else { return nil }
+
+    array[head] = nil
+    head += 1
+
+    if capacity >= originalCapacity && head >= capacity*2 {
+      let amountToRemove = capacity + capacity/2
+      array.removeFirst(amountToRemove)
+      head -= amountToRemove
+      capacity /= 2
+    }
+    return element
+  }
+

Recall that capacity is the original number of empty places at the front of the queue. If the head has advanced more to the right than twice the capacity, then it's time to trim off a bunch of these empty spots. We reduce it to about 25%.

+
+

Note: The deque will keep at least its original capacity by comparing capacity to originalCapacity.

+
+

For example, this:

+
[ x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, x, 1, 2, 3 ]
+                                |                             |
+                                capacity                      head
+
+

becomes after trimming:

+
[ x, x, x, x, x, 1, 2, 3 ]
+                 |
+                 head
+                 capacity
+
+

This way we can strike a balance between fast enqueuing and dequeuing at the front and keeping the memory requirements reasonable.

+
+

Note: We don't perform trimming on very small arrays. It's not worth it for saving just a few bytes of memory.

+
+

See also

+

Other ways to implement deque are by using a doubly linked list, a circular buffer, or two stacks facing opposite directions.

+

A fully-featured deque implementation in Swift

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Dijkstra Algorithm/index.html b/Dijkstra Algorithm/index.html new file mode 100644 index 000000000..c2ea02bcc --- /dev/null +++ b/Dijkstra Algorithm/index.html @@ -0,0 +1,533 @@ + + + Dijkstra Algorithm + + + +

Weighted graph general concepts

+

Every weighted graph should contain:

+
    +
  1. Vertices/Nodes (I will use "vertex" in this readme).
    2. +
+

+
    +
  1. Edges connecting vertices. Let's add some edges to our graph. For simplicity let's create directed graph for now. Directed means that edge has a direction, i.e. vertex, where it starts and vertex, where it ends. But remember a VERY IMPORTANT thing: +
      +
    • All undirected graphs can be viewed as a directed graph.
      • +
    • A directed graph is undirected if and only if every edge is paired with an edge going in the opposite direction.
      • +
    +
    2. +
+

+
    +
  1. Weights for every edge.
    2. +
+

+

Final result.
+Directed weighted graph:

+

+

Undirected weighted graph:

+

+

And once again: An undirected graph it is a directed graph with every edge paired with an edge going in the opposite direction. This statement is clear on the image above.

+

Great! Now we are familiar with general concepts about graphs.

+

The Dijkstra's algorithm

+

This algorithm was invented in 1956 by Edsger W. Dijkstra.

+

It can be used when you have one source vertex and want to find the shortest paths to ALL other vertices in the graph.

+

The best example is a road network. If you want to find the shortest path from your house to your job or if you want to find the closest store to your house then it is time for the Dijkstra's algorithm.

+

The algorithm repeats following cycle until all vertices are marked as visited.
+Cycle:

+
    +
  1. From the non-visited vertices the algorithm picks a vertex with the shortest path length from the start (if there are more than one vertex with the same shortest path value then algorithm picks any of them)
    2. +
  2. The algorithm marks picked vertex as visited.
    3. +
  3. The algorithm checks all of its neighbours. If the current vertex path length from the start plus an edge weight to a neighbour less than the neighbour current path length from the start than it assigns new path length from the start to the neighbour.
    +When all vertices are marked as visited, the algorithm's job is done. Now, you can see the shortest path from the start for every vertex by pressing the one you are interested in.
    4. +
+

I have created VisualizedDijkstra.playground game/tutorial to improve your understanding of the algorithm's flow. Besides, below is step by step algorithm's description.

+

A short sidenote. The Swift Algorithm Club also contains the A* algorithm, which essentially is a faster version of Dijkstra's algorithm for which the only extra prerequisite is you have to know where the destination is located.

+

Example

+

Let's imagine that you want to go to the shop. Your house is A vertex and there are 4 possible stores around your house. How to find the closest one/ones? Luckily, you have a graph that connects your house with all these stores. So, you know what to do :)

+

Initialisation

+

When the algorithm starts to work initial graph looks like this:

+

+

The table below represents graph state:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedFFFFF
Path Length From Startinfinfinfinfinf
Path Vertices From Start[ ][ ][ ][ ][ ]
+
+

inf is equal infinity which basically means that algorithm doesn't know how far away is this vertex from start one.

+
+
+

F states for False

+
+
+

T states for True

+
+

To initialize our graph we have to set source vertex path length from source vertex to 0 and append itself to path vertices from start.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedFFFFF
Path Length From Start0infinfinfinf
Path Vertices From Start[A][ ][ ][ ][ ]
+

Great, now our graph is initialised and we can pass it to the Dijkstra's algorithm, let's start!

+

Let's follow the algorithm's cycle and pick the first vertex which neighbours we want to check.
+All our vertices are not visited but there is only one has the smallest path length from start. It is A. This vertex is the first one which neighbors we will check.
+First of all, set this vertex as visited.

+

A.visited = true

+

+

After this step graph has this state:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTFFFF
Path Length From Start0infinfinfinf
Path Vertices From Start[A][ ][ ][ ][ ]
+

Step 1

+

Then we check all of its neighbours.
+If checking vertex path length from start + edge weight is smaller than neighbour's path length from start then we set neighbour's path length from start new value and append to its pathVerticesFromStart array new vertex: checkingVertex. Repeat this action for every vertex.

+

for clarity:

+
if (A.pathLengthFromStart + AB.weight) < B.pathLengthFromStart {
+    B.pathLengthFromStart = A.pathLengthFromStart + AB.weight
+    B.pathVerticesFromStart = A.pathVerticesFromStart
+    B.pathVerticesFromStart.append(B)
+}
+

And now our graph looks like this one:

+

+

And its state is here:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTFFFF
Path Length From Start03inf1inf
Path Vertices From Start[A][A, B][ ][A, D][ ]
+

Step 2

+

From now we repeat all actions again and fill our table with new info!

+

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTFFTF
Path Length From Start03inf12
Path Vertices From Start[A][A, B][ ][A, D][A, D, E]
+

Step 3

+

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTFFTT
Path Length From Start031112
Path Vertices From Start[A][A, B][A, D, E, C][A, D][A, D, E ]
+

Step 4

+

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTTFTT
Path Length From Start03812
Path Vertices From Start[A][A, B][A, B, C][A, D][A, D, E ]
+

Step 5

+

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ABCDE
VisitedTTTTT
Path Length From Start03812
Path Vertices From Start[A][A, B][A, B, C][A, D][A, D, E ]
+

Code implementation

+

First of all, let’s create class that will describe any Vertex in the graph.
+It is pretty simple

+
open class Vertex {
+
+    //Every vertex should be unique that's why we set up identifier
+    open var identifier: String
+
+    //For Dijkstra every vertex in the graph should be connected with at least one other vertex. But there can be some usecases
+    //when you firstly initialize all vertices without neighbours. And then on next iteration you set up their neighbours. So, initially neighbours is an empty array.
+    //Array contains tuples (Vertex, Double). Vertex is a neighbour and Double is as edge weight to that neighbour.
+    open var neighbours: [(Vertex, Double)] = []
+
+    //As it was mentioned in the algorithm description, default path length from start for all vertices should be as much as possible.
+    //It is var because we will update it during the algorithm execution.
+    open var pathLengthFromStart = Double.infinity
+
+    //This array contains vertices which we need to go through to reach this vertex from starting one
+    //As with path length from start, we will change this array during the algorithm execution.
+    open var pathVerticesFromStart: [Vertex] = []
+
+    public init(identifier: String) {
+        self.identifier = identifier
+    }
+
+    //This function let us use the same array of vertices again and again to calculate paths with different starting vertex.
+    //When we will need to set new starting vertex and recalculate paths then we will simply clear graph vertices' cashes.
+    open func clearCache() {
+        pathLengthFromStart = Double.infinity
+        pathVerticesFromStart = []
+    }
+}
+

As every vertex should be unique it is useful to make them Hashable and according Equatable. We use an identifier for this purposes.

+
extension Vertex: Hashable {
+    open var hashValue: Int {
+        return identifier.hashValue
+    }
+}
+
+extension Vertex: Equatable {
+    public static func ==(lhs: Vertex, rhs: Vertex) -> Bool {
+        return lhs.hashValue == rhs.hashValue
+    }
+}
+

We've created a base for our algorithm. Now let's create a house :)
+Dijkstra's realisation is really straightforward.

+
public class Dijkstra {
+    //This is a storage for vertices in the graph.
+    //Assuming that our vertices are unique we can use Set instead of array. This approach will bring some benefits later.
+    private var totalVertices: Set<Vertex>
+
+    public init(vertices: Set<Vertex>) {
+        totalVertices = vertices
+    }
+
+    //Remember clearCache function in the Vertex class implementation?
+    //This is just a wrapper that cleans cache for all stored vertices.
+    private func clearCache() {
+        totalVertices.forEach { $0.clearCache() }
+    }
+
+    public func findShortestPaths(from startVertex: Vertex) {
+	//Before we start searching the shortest path from startVertex,
+	//we need to clear vertices cache just to be sure that out graph is clean.
+	//Remember that every Vertex is a class and classes are passed by reference.
+	//So whenever you change vertex outside of this class it will affect this vertex inside totalVertices Set
+        clearCache()
+	//Now all our vertices have Double.infinity pathLengthFromStart and an empty pathVerticesFromStart array.
+
+	//The next step in the algorithm is to set startVertex pathLengthFromStart and pathVerticesFromStart
+        startVertex.pathLengthFromStart = 0
+        startVertex.pathVerticesFromStart.append(startVertex)
+
+	//Here starts the main part. We will use while loop to iterate through all vertices in the graph.
+	//For this purpose we define currentVertex variable which we will change in the end of each while cycle.
+        var currentVertex: Vertex? = startVertex
+
+        while let vertex = currentVertex {
+
+    	    //Next line of code is an implementation of setting vertex as visited.
+    	    //As it has been said, we should check only unvisited vertices in the graph,
+	    //So why don't just delete it from the set? This approach let us skip checking for *"if !vertex.visited then"*
+            totalVertices.remove(vertex)
+
+	    //filteredNeighbours is an array that contains current vertex neighbours which aren't yet visited
+            let filteredNeighbours = vertex.neighbours.filter { totalVertices.contains($0.0) }
+
+	    //Let's iterate through them
+            for neighbour in filteredNeighbours {
+		//These variable are more representative, than neighbour.0 or neighbour.1
+                let neighbourVertex = neighbour.0
+                let weight = neighbour.1
+
+		//Here we calculate new weight, that we can offer to neighbour.
+                let theoreticNewWeight = vertex.pathLengthFromStart + weight
+
+		//If it is smaller than neighbour's current pathLengthFromStart
+		//Then we perform this code
+                if theoreticNewWeight < neighbourVertex.pathLengthFromStart {
+
+		    //set new pathLengthFromStart
+                    neighbourVertex.pathLengthFromStart = theoreticNewWeight
+
+		    //set new pathVerticesFromStart
+                    neighbourVertex.pathVerticesFromStart = vertex.pathVerticesFromStart
+
+		    //append current vertex to neighbour's pathVerticesFromStart
+                    neighbourVertex.pathVerticesFromStart.append(neighbourVertex)
+                }
+            }
+
+	    //If totalVertices is empty, i.e. all vertices are visited
+	    //Than break the loop
+            if totalVertices.isEmpty {
+                currentVertex = nil
+                break
+            }
+
+	    //If loop is not broken, than pick next vertex for checkin from not visited.
+	    //Next vertex pathLengthFromStart should be the smallest one.
+            currentVertex = totalVertices.min { $0.pathLengthFromStart < $1.pathLengthFromStart }
+        }
+    }
+}
+

That's all! Now you can check this algorithm in the playground. On the main page there is a code for creating a random graph.

+

Also there is a VisualizedDijkstra.playground. Use it to figure out the algorithm's flow in real (slowed :)) time.

+

It is up to you how to implement some specific parts of the algorithm, you can use Array instead of Set, add visited property to Vertex or you can create some local totalVertices Array/Set inside func findShortestPaths(from startVertex: Vertex) to keep totalVertices Array/Set unchanged. This is a general explanation with one possible implementation :)

+

About this repository

+

This repository contains two playgrounds:

+ +

Demo video

+

Click the link: YouTube

+

Credits

+

WWDC 2017 Scholarship Project (Rejected) created by Taras Nikulin

+ + diff --git a/DiningPhilosophers/index.html b/DiningPhilosophers/index.html new file mode 100644 index 000000000..728be7f08 --- /dev/null +++ b/DiningPhilosophers/index.html @@ -0,0 +1,41 @@ + + + DiningPhilosophers + + + +

Dining Philosophers

+

The dining philosophers problem Algorithm implemented in Swift (concurrent algorithm design to illustrate synchronization issues and techniques for resolving them using GCD and Semaphore in Swift)

+

Written for Swift Algorithm Club by Jacopo Mangiavacchi

+

Introduction

+

In computer science, the dining philosophers problem is often used in the concurrent algorithm design to illustrate synchronization issues and techniques for resolving them.

+

It was originally formulated in 1965 by Edsger Dijkstra as a student exam exercise, presented in terms of computers competing for access to tape drive peripherals. Soon after, Tony Hoare gave the problem its present formulation.

+

This Swift implementation is based on the Chandy/Misra solution, and it uses the GCD Dispatch and Semaphores on the Swift cross platform.

+

Problem statement

+

Five silent philosophers sit at a round table with bowls of spaghetti. Forks are placed between each pair of adjacent philosophers.

+

Each philosopher must alternately think and eat. A philosopher can only eat spaghetti when they have both left and right forks. Since each fork can be held by only one philosopher, a philosopher can use the fork only if it is not being used by another philosopher. When a philosopher finishes eating, they need to put down both forks so that the forks become available to others. A philosopher can take the fork on their right or the one on their left as they become available, but they cannot start eating before getting both forks.

+

Eating is not limited by the remaining amounts of spaghetti or stomach space; an infinite supply and an infinite demand are assumed.

+

The problem is how to design a discipline of behavior (a concurrent algorithm) such that no philosopher will starve; i.e., each can forever continue to alternate between eating and thinking, assuming that no philosopher can know when others may want to eat or think.

+

This is an illustration of a dining table:

+

Dining Philosophers table

+

Solution

+

There are different solutions for this classic algorithm, and this Swift implementation is based on the Chandy/Misra solution. This implementation allows agents to contend for an arbitrary number of resources in a completely distributed scenario with no need for a central authority to control the locking and serialization of resources.

+

However, this solution violates the requirement that "the philosophers do not speak to each other" (due to the request messages).

+

Description

+

For every pair of philosophers contending for a resource, create a fork and give it to the philosopher with the lower ID (n for agent Pn). Each fork can either be dirty or clean. Initially, all forks are dirty.
+When a philosopher wants to use a set of resources (i.e. eat), said philosopher must obtain the forks from their contending neighbors. The philospher send a message for all such forks needed. When a philosopher with a fork receives a request message, they keep the fork if it is clean, but give it up when it is dirty. If the philosopher sends the fork over, they clean the fork before doing so.
+After a philosopher is done eating, all their forks become dirty. If another philosopher had previously requested one of the forks, the philosopher that has just finished eating cleans the fork and sends it.
+This solution also allows for a large degree of concurrency, and it will solve an arbitrarily large problem.

+

In addition, it solves the starvation problem. The clean / dirty labels give a preference to the most "starved" processes and a disadvantage to processes that have just "eaten". One could compare their solution to one where philosophers are not allowed to eat twice in a row without letting others use the forks in between. The Chandy and Misra's solution is more flexible but has an element tending in that direction.

+

Based on the Chandy and Misra's analysis, a system of preference levels is derived from the distribution of the forks and their clean/dirty states. This system may describe an acyclic graph, and if so, the solution's protocol cannot turn that graph into a cyclic one. This guarantees that deadlock cannot occur. However, if the system is initialized to a perfectly symmetric state, such as all philosophers holding their left side forks, then the graph is cyclic at the outset, and the solution cannot prevent a deadlock. Initializing the system so that philosophers with lower IDs have dirty forks ensures the graph is initially acyclic.

+

Swift implementation

+

This Swift 3.0 implementation of the Chandy/Misra solution is based on the GCD and Semaphore technique that can be built on both macOS and Linux.

+

The code is based on a ForkPair struct used for holding an array of DispatchSemaphore and a Philosopher struct for associate a couple of forks to each Philosopher.

+

The ForkPair DispatchSemaphore static array is used for waking the neighbour Philosophers any time a fork pair is put down on the table.

+

A background DispatchQueue is then used to let any Philosopher run asyncrounosly on the background, and a global DispatchSemaphore is used to keep the main thread on wait forever and let the Philosophers continue forever in their alternate think and eat cycle.

+

See also

+

Dining Philosophers on Wikipedia https://en.wikipedia.org/wiki/Dining_philosophers_problem

+

Written for Swift Algorithm Club by Jacopo Mangiavacchi
+Swift 4.2 check by Bruno Scheele

+ + diff --git a/Egg Drop Problem/index.html b/Egg Drop Problem/index.html new file mode 100644 index 000000000..4cb8cb096 --- /dev/null +++ b/Egg Drop Problem/index.html @@ -0,0 +1,67 @@ + + + Egg Drop Problem + + + +

Egg Drop

+

The egg drop problem is an interview question popularized by Google. The premise is simple; You're given a task to evaluate the shatter resistance of unknown objects by dropping them at a certain height. For simplicity, you test this by going inside a multi-story building and performing tests by dropping the objects out the window and onto the ground:

+

building with eggs being dropped

+

Your goal is to find out the minimum height that causes the object to shatter. Consider the trivial case you're given 1 object to obtain the results with. Since you've only got one sample for testing, you need to play it safe by performing drop tests starting with the bottom floor and working your way up:

+

dropping from first floor

+

If the object is incredibly resilient, and you may need to do the testing on the world's tallest building - the Burj Khalifa. With 163 floors, that's a lot of climbing. Let's assume you complain, and your employer hears your plight. You are now given several samples to work with. How can you make use of these extra samples to expedite your testing process? The problem for this situation is popularized as the egg drop problem.

+

Description

+

You're in a building with m floors and you are given n eggs. What is the minimum number of attempts it will take to find out the floor that breaks the egg?

+

For convenience, here are a few rules to keep in mind:

+ +

Solution

+ +

We store all the solutions in a 2D array. Where rows represents number of eggs and columns represent number of floors.

+

First, we set base cases:

+
    +
  1. If there's only one egg, it takes as many attempts as number of floors
    2. +
  2. If there are two eggs and one floor, it takes one attempt
    3. +
+
for var floorNumber in (0..<(numberOfFloors+1)){
+eggFloor[1][floorNumber] = floorNumber      //base case 1: if there's only one egg, it takes 'numberOfFloors' attempts
+}
+
+eggFloor[2][1] = 1 //base case 2: if there are two eggs and one floor, it takes one attempt
+

When we drop an egg from a floor 'floorNumber', there can be two cases (1) The egg breaks (2) The egg doesn’t break.

+
    +
  1. If the egg breaks after dropping from 'visitingFloorth' floor, then we only need to check for floors lower than 'visitingFloor' with remaining eggs; so the problem reduces to 'visitingFloor'-1 floors and 'eggNumber'-1 eggs.
    2. +
  2. If the egg doesn’t break after dropping from the 'visitingFloorth' floor, then we only need to check for floors higher than 'visitingFloor'; so the problem reduces to floors-'visitingFloor' floors and 'eggNumber' eggs.
    3. +
+

Since we need to minimize the number of trials in worst case, we take the maximum of two cases. We consider the max of above two cases for every floor and choose the floor which yields minimum number of trials.

+

We find the answer based on the base cases and previously found answers as follows.

+
attempts = 1 + max(eggFloor[eggNumber-1][floors-1], eggFloor[eggNumber][floorNumber-floors])//we add one taking into account the attempt we're taking at the moment
+
+if attempts < eggFloor[eggNumber][floorNumber]{ //finding the min
+    eggFloor[eggNumber][floorNumber] = attempts;
+}
+

Example

+

Let's assume we have 2 eggs and 2 floors.

+
    +
  1. We drop one egg from the first floor. If it breaks, then we get the answer. If it doesn't we'll have 2 eggs and 1 floors to work with.
    +attempts = 1 + maximum of 0(got the answer) and eggFloor[2][1] (base case 2 which gives us 1)
    +attempts = 1 + 1 = 2
    2. +
  2. We drop one egg from the second floor. If it breaks, we'll have 1 egg and 1 floors to work with. If it doesn't, we'll get the answer.
    +attempts = 1 + maximum of eggFloor[1][1](base case 1 which gives us 1) and 0(got the answer)
    +attempts = 1 + 1 = 2
    3. +
  3. Finding the minimum of 2 and 2 gives us 2, so the answer is 2.
    +2 is the minimum number of attempts it will take to find out from which floor egg will break.
    4. +
+

Written for the Swift Algorithm Club by Arkalyk Akash. Revisions and additions by Kelvin Lau

+ + diff --git a/Encode and Decode Tree/index.html b/Encode and Decode Tree/index.html new file mode 100644 index 000000000..5a687fc4b --- /dev/null +++ b/Encode and Decode Tree/index.html @@ -0,0 +1,183 @@ + + + Encode and Decode Tree + + + +

Encode and Decode Binary Tree

+
+

Note: The prerequisite for this article is an understanding of how binary trees work.

+
+

Trees are complex structures. Unlike linear collections such as arrays or linked lists, trees are non-linear and each element in a tree has positional information such as the parent-child relationship between nodes. When you want to send a tree structure to your backend, you need to send the data of each node, and a way to represent the parent-child relationship for each node.

+

Your strategy in how you choose to represent this information is called your encoding strategy. The opposite of that - changing your encoded data back to its original form - is your decoding strategy.

+

There are many ways to encode a tree and decode a tree. The important thing to keep in mind is that encoding and decoding strategies are closely related. The way you choose to encode a tree directly affects how you might decode a tree.

+

Encoding and decoding are synonyms to serializing and deserializing trees.

+

As a reference, the following code represents the typical Node type of a binary tree:

+
class BinaryNode<Element: Comparable> {
+  var data: Element
+  var leftChild: BinaryNode?
+  var rightChild: BinaryNode?
+
+  // ... (rest of the implementation)
+}
+

Your encoding and decoding methods will reside in the BinaryNodeEncoder and BinaryNodeDecoder classes:

+
class BinaryNodeCoder {
+ 
+  // transforms nodes into string representation
+  func encode<T>(_ node: BinaryNode<T>) throws -> String where T: Encodable {
+    
+  }
+  
+  // transforms string into `BinaryNode` representation
+  func decode<T>(from string: String) 
+    throws -> BinaryNode<T> where T: Decodable {
+  
+  }
+}
+

Encoding

+

As mentioned before, there are different ways to do encoding. For no particular reason, you'll opt for the following rules:

+
    +
  1. The result of the encoding will be a String object.
    2. +
  2. You'll encode using pre-order traversal.
    3. +
+

Here's an example of this operation in code:

+
fileprivate extension BinaryNode {
+  
+  // 1
+  func preOrderTraversal(visit: (Element?) throws -> ()) rethrows {
+    try visit(data)
+    
+    if let leftChild = leftChild {
+      try leftChild.preOrderTraversal(visit: visit)
+    } else {
+      try visit(nil)
+    }
+    
+    if let rightChild = rightChild {
+      try rightChild.preOrderTraversal(visit: visit)
+    } else {
+      try visit(nil)
+    }
+  }
+}
+
+class BinaryNodeCoder {
+
+  // 2
+  private var separator: String { return "," }
+  
+  // 3
+  private var nilNode: String { return "X" }
+  
+  // 4
+  func encode<T>(_ node: BinaryNode<T>) -> String {
+    var str = ""
+    node.preOrderTraversal { data in
+      if let data = data {
+        let string = String(describing: data)
+        str.append(string)
+      } else {
+        str.append(nilNode)
+      }
+      str.append(separator)
+    }
+    return str
+  }
+  
+  // ...
+}
+

Here's a high level overview of the above code:

+
    +
  1. +

    separator is a way to distinguish the nodes in a string. To illustrate its importance, consider the following encoded string "banana". How did the tree structure look like before encoding? Without the separator, you can't tell.

    +
    2. +
  2. +

    nilNode is used to identify empty children. This a necesssary piece of information to retain in order to rebuild the tree later.

    +
    3. +
  3. +

    encode returns a String representation of the BinaryNode. For example: "ba,nana,nil" represents a tree with two nodes - "ba" and "nana" - in pre-order format.

    +
    4. +
+

Decoding

+

Your decoding strategy is the exact opposite of your encoding strategy. You'll take an encoded string, and turn it back into your binary tree.

+

Your encoding strategy followed the following rules:

+
    +
  1. The result of the encoding will be a String object.
    2. +
  2. You'll encode using pre-order traversal.
    3. +
+

The implementation also added a few important details:

+ +

These details will shape your decode operation. Here's a possible implementation:

+
class BinaryNodeCoder {
+
+  // ...
+  
+  // 1
+  func decode<T>(_ string: String) -> BinaryNode<T>? {
+    let components = encoded.lazy.split(separator: separator).reversed().map(String.init)
+    return decode(from: components)
+  }
+  
+  // 2
+  private func decode(from array: inout [String]) -> BinaryNode<String>? {
+    guard !array.isEmpty else { return nil }
+    let value = array.removeLast()
+    guard value != "\(nilNode)" else { return nil }
+    
+    let node = AVLNode<String>(value: value)
+    node.leftChild = decode(from: &array)
+    node.rightChild = decode(from: &array)
+    return node
+  }
+}
+

Here's a high level overview of the above code:

+
    +
  1. +

    Takes a String, and uses split to partition the contents of string into an array based on the separator defined in the encoding step. The result is first reversed, and then mapped to a String. The reverse step is an optimization for the next function, allowing us to use array.removeLast() instead of array.removeFirst().

    +
    2. +
  2. +

    Using an array as a stack, you recursively decode each node. The array keeps track of sequence of nodes and progress.

    +
    3. +
+

Here's an example output of a tree undergoing the encoding and decoding process:

+
Original Tree
+
+  ┌──8423
+ ┌──8391
+ │ └──nil
+┌──7838
+│ │ ┌──4936
+│ └──3924
+│  └──2506
+830
+│ ┌──701
+└──202
+ └──169
+
+Encoded tree: 830,202,169,X,X,701,X,X,7838,3924,2506,X,X,4936,X,X,8391,X,8423,X,X,
+
+Decoded tree
+
+  ┌──8423
+ ┌──8391
+ │ └──nil
+┌──7838
+│ │ ┌──4936
+│ └──3924
+│  └──2506
+830
+│ ┌──701
+└──202
+ └──169
+
+

Notice the original tree and decoded tree are identical.

+

Further Reading & References

+ +

Written for the Swift Algorithm Club by Kai Chen & Kelvin Lau

+ + diff --git a/Fixed Size Array/index.html b/Fixed Size Array/index.html new file mode 100644 index 000000000..e47c3973a --- /dev/null +++ b/Fixed Size Array/index.html @@ -0,0 +1,110 @@ + + + Fixed Size Array + + + +

Fixed-Size Arrays

+

Early programming languages didn't have very fancy arrays. You'd create the array with a specific size and from that moment on it would never grow or shrink. Even the standard arrays in C and Objective-C are still of this type.

+

When you define an array like so,

+
int myArray[10];
+
+

the compiler allocates one contiguous block of memory that can hold 40 bytes (assuming an int is 4 bytes):

+

An array with room for 10 elements

+

That's your array. It will always be this size. If you need to fit more than 10 elements, you're out of luck... there is no room for it.

+

To get an array that grows when it gets full you need to use a dynamic array object such as NSMutableArray in Objective-C or std::vector in C++, or a language like Swift whose arrays increase their capacity as needed.

+

A major downside of the old-style arrays is that they need to be big enough or you run out of space. But if they are too big you're wasting memory. And you need to be careful about security flaws and crashes due to buffer overflows. In summary, fixed-size arrays are not flexible and they leave no room for error.

+

That said, I like fixed-size arrays because they are simple, fast, and predictable.

+

The following operations are typical for an array:

+ +

For a fixed-size array, appending is easy as long as the array isn't full yet:

+

Appending a new element

+

Looking up by index is also quick and easy:

+

Indexing the array

+

These two operations have complexity O(1), meaning the time it takes to perform them is independent of the size of the array.

+

For an array that can grow, appending is more involved: if the array is full, new memory must be allocated and the old contents copied over to the new memory buffer. On average, appending is still an O(1) operation, but what goes on under the hood is less predictable.

+

The expensive operations are inserting and deleting. When you insert an element somewhere that's not at the end, it requires moving up the remainder of the array by one position. That involves a relatively costly memory copy operation. For example, inserting the value 7 in the middle of the array:

+

Insert requires a memory copy

+

If your code was using any indexes into the array beyond the insertion point, these indexes are now referring to the wrong objects.

+

Deleting requires a copy the other way around:

+

Delete also requires a memory copy

+

This, by the way, is also true for NSMutableArray or Swift arrays. Inserting and deleting are O(n) operations -- the larger the array the more time it takes.

+

Fixed-size arrays are a good solution when:

+
    +
  1. You know beforehand the maximum number of elements you'll need. In a game this could be the number of sprites that can be active at a time. It's not unreasonable to put a limit on this. (For games it's a good idea to allocate all the objects you need in advance anyway.)
    2. +
  2. It is not necessary to have a sorted version of the array, i.e. the order of the elements does not matter.
    3. +
+

If the array does not need to be sorted, then an insertAt(index) operation is not needed. You can simply append any new elements to the end, until the array is full.

+

The code for adding an element becomes:

+
func append(_ newElement: T) {
+  if count < maxSize {
+    array[count] = newElement
+    count += 1
+  }
+}
+

The count variable keeps track of the size of the array and can be considered the index just beyond the last element. That's the index where you'll insert the new element.

+

Determining the number of elements in the array is just a matter of reading the count variable, a O(1) operation.

+

The code for removing an element is equally simple:

+
func removeAt(index: Int) {
+  count -= 1
+  array[index] = array[count]
+}
+

This copies the last element on top of the element you want to remove, and then decrements the size of the array.

+

Deleting just means copying one element

+

This is why the array is not sorted. To avoid an expensive copy of a potentially large portion of the array we copy just one element, but that does change the order of the elements.

+

There are now two copies of element 6 in the array, but what was previously the last element is no longer part of the active array. It's just junk data -- the next time you append an new element, this old version of 6 will be overwritten.

+

Under these two constraints -- a limit on the number of elements and an unsorted array -- fixed-size arrays are still perfectly suitable for use in modern software.

+

Here is an implementation in Swift:

+
struct FixedSizeArray<T> {
+  private var maxSize: Int
+  private var defaultValue: T
+  private var array: [T]
+  private (set) var count = 0
+  
+  init(maxSize: Int, defaultValue: T) {
+    self.maxSize = maxSize
+    self.defaultValue = defaultValue
+    self.array = [T](repeating: defaultValue, count: maxSize)
+  }
+  
+  subscript(index: Int) -> T {
+    assert(index >= 0)
+    assert(index < count)
+    return array[index]
+  }
+  
+  mutating func append(_ newElement: T) {
+    assert(count < maxSize)
+    array[count] = newElement
+    count += 1
+  }
+  
+  mutating func removeAt(index: Int) -> T {
+    assert(index >= 0)
+    assert(index < count)
+    count -= 1
+    let result = array[index]
+    array[index] = array[count]
+    array[count] = defaultValue
+    return result
+  }
+  
+  mutating func removeAll() {
+    for i in 0..<count {
+      array[i] = defaultValue
+    }
+    count = 0
+  }
+}
+

When creating the array, you specify the maximum size and a default value:

+
var a = FixedSizeArray(maxSize: 10, defaultValue: 0)
+

Note that removeAt(index: Int) overwrites the last element with this defaultValue to clean up the "junk" object that gets left behind. Normally it wouldn't matter to leave that duplicate object in the array, but if it's a class or a struct it may have strong references to other objects and it's good boyscout practice to zero those out.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Fizz Buzz/index.html b/Fizz Buzz/index.html new file mode 100644 index 000000000..d21c7a616 --- /dev/null +++ b/Fizz Buzz/index.html @@ -0,0 +1,192 @@ + + + Fizz Buzz + + + +

Fizz Buzz

+

Fizz buzz is a group word game for children to teach them about division. Players take turns to count incrementally, replacing any number divisible by three with the word "fizz", and any number divisible by five with the word "buzz".

+

Fizz buzz has been used as an interview screening device for computer programmers.

+

Example

+

A typical round of fizz buzz:

+

1, 2, Fizz, 4, Buzz, Fizz, 7, 8, Fizz, Buzz, 11, Fizz, 13, 14, Fizz Buzz, 16, 17, Fizz, 19, Buzz, Fizz, 22, 23, Fizz, Buzz, 26, Fizz, 28, 29, Fizz Buzz, 31, 32, Fizz, 34, Buzz, Fizz, ...

+

Modulus Operator

+

The modulus operator % is the key to solving fizz buzz.

+

The modulus operator returns the remainder after an integer division. Here is an example of the modulus operator:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
DivisionDivision ResultModulusModulus Result
1 / 30 with a remainder of 31 % 31
5 / 31 with a remainder of 25 % 32
16 / 35 with a remainder of 116 % 31
+

A common approach to determine if a number is even or odd is to use the modulus operator:

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
ModulusResultSwift CodeSwift Code ResultComment
6 % 20let isEven = (number % 2 == 0)trueIf a number is divisible by 2 it is even
5 % 21let isOdd = (number % 2 != 0)trueIf a number is not divisible by 2 it is odd
+

Solving fizz buzz

+

Now we can use the modulus operator % to solve fizz buzz.

+

Finding numbers divisible by three:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModulusModulus ResultSwift CodeSwift Code Result
1 % 311 % 3 == 0false
2 % 322 % 3 == 0false
3 % 303 % 3 == 0true
4 % 314 % 3 == 0false
+

Finding numbers divisible by five:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModulusModulus ResultSwift CodeSwift Code Result
1 % 511 % 5 == 0false
2 % 522 % 5 == 0false
3 % 533 % 5 == 0false
4 % 544 % 5 == 0false
5 % 505 % 5 == 0true
6 % 516 % 5 == 0false
+

The code

+

Here is a simple implementation in Swift:

+
func fizzBuzz(_ numberOfTurns: Int) {
+  for i in 1...numberOfTurns {
+    var result = ""
+
+    if i % 3 == 0 {
+      result += "Fizz"
+    }
+
+    if i % 5 == 0 {
+      result += (result.isEmpty ? "" : " ") + "Buzz"
+    }
+
+    if result.isEmpty {
+      result += "\(i)"
+    }
+
+    print(result)
+  }
+}
+

Put this code in a playground and test it like so:

+
fizzBuzz(15)
+

This will output:

+
1, 2, Fizz, 4, Buzz, Fizz, 7, 8, Fizz, Buzz, 11, Fizz, 13, 14, Fizz Buzz
+
+

See also

+

Fizz buzz on Wikipedia

+

Written by Chris Pilcher

+ + diff --git a/GCD/index.html b/GCD/index.html new file mode 100644 index 000000000..d47a93e69 --- /dev/null +++ b/GCD/index.html @@ -0,0 +1,136 @@ + + + GCD + + + +

Greatest Common Divisor

+

The greatest common divisor (or Greatest Common Factor) of two numbers a and b is the largest positive integer that divides both a and b without a remainder. The GCD.swift file contains three different algorithms of how to calculate the greatest common divisor.
+For example, gcd(39, 52) = 13 because 13 divides 39 (39 / 13 = 3) as well as 52 (52 / 13 = 4). But there is no larger number than 13 that divides them both.
+You've probably had to learn about this in school at some point. :-)

+

You probably won't need to use the GCD or LCM in any real-world problems, but it's cool to play around with this ancient algorithm. It was first described by Euklid in his Elements around 300 BC. Rumor has it that he discovered this algorithm while he was hacking on his Commodore 64.

+

Different Algorithms

+

This example includes three different algorithms to find the same result.

+

Iterative Euklidean

+

The laborious way to find the GCD of two numbers is to first figure out the factors of both numbers, then take the greatest number they have in common. The problem is that factoring numbers is quite difficult, especially when they get larger. (On the plus side, that difficulty is also what keeps your online payments secure.)
+There is a smarter way to calculate the GCD: Euklid's algorithm. The big idea here is that,

+
gcd(a, b) = gcd(b, a % b)
+
+

where a % b calculates the remainder of a divided by b.

+

Here is an implementation of this idea in Swift:

+
func gcdIterativeEuklid(_ m: Int, _ n: Int) -> Int {
+    var a: Int = 0
+    var b: Int = max(m, n)
+    var r: Int = min(m, n)
+
+    while r != 0 {
+        a = b
+        b = r
+        r = a % b
+    }
+    return b
+}
+

Put it in a playground and try it out with these examples:

+
gcd(52, 39, using: gcdIterativeEuklid)        // 13
+gcd(228, 36, using: gcdIterativeEuklid)       // 12
+gcd(51357, 3819, using: gcdIterativeEuklid)   // 57
+

Recursive Euklidean

+

Here is a slightly different implementation of Euklid's algorithm. Unlike the first version this doesn't use a while loop, but leverages recursion.

+
func gcdRecursiveEuklid(_ m: Int, _ n: Int) -> Int {
+    let r: Int = m % n
+    if r != 0 {
+        return gcdRecursiveEuklid(n, r)
+    } else {
+        return n
+    }
+}
+

Put it in a playground and compare it with the results of the iterative Eulidean gcd. They should return the same results:

+
gcd(52, 39, using: gcdRecursiveEuklid)        // 13
+gcd(228, 36, using: gcdRecursiveEuklid)       // 12
+gcd(51357, 3819, using: gcdRecursiveEuklid)   // 57
+

The max() and min() at the top of the function make sure we always divide the larger number by the smaller one.

+

Let's step through the third example using the recursive Euklidean algorithm here:

+
gcd(51357, 3819)
+
+

According to Euklid's rule, this is equivalent to,

+
gcd(3819, 51357 % 3819) = gcd(3819, 1710)
+
+

because the remainder of 51357 % 3819 is 1710. If you work out this division you get 51357 = (13 * 3819) + 1710 but we only care about the remainder part.

+

So gcd(51357, 3819) is the same as gcd(3819, 1710). That's useful because we can keep simplifying:

+
gcd(3819, 1710) = gcd(1710, 3819 % 1710) =
+gcd(1710, 399)  = gcd(399, 1710 % 399)   =
+gcd(399, 114)   = gcd(114, 399 % 114)    =
+gcd(114, 57)    = gcd(57, 114 % 57)      =
+gcd(57, 0)
+
+

And now can't divide any further. The remainder of 114 / 57 is zero because 114 = 57 * 2 exactly. That means we've found the answer:

+
gcd(3819, 51357) = gcd(57, 0) = 57
+
+

So in each step of Euklid's algorithm the numbers become smaller and at some point it ends when one of them becomes zero.

+

By the way, it's also possible that two numbers have a GCD of 1. They are said to be relatively prime. This happens when there is no number that divides them both, for example:

+
gcd(841, 299)     // 1
+

Binary Recursive Stein

+

The binary GCD algorithm, also known as Stein's algorithm, is an algorithm that computes the greatest common divisor of two nonnegative integers. Stein's algorithm is very similar to the recursive Eulidean algorithm, but uses arithmetical operations, which are simpler for computers to perform, than the conventional Euclidean algorithm does. It replaces division with arithmetic shifts, comparisons, and subtraction.

+
func gcdBinaryRecursiveStein(_ m: Int, _ n: Int) -> Int {
+    if let easySolution = findEasySolution(m, n) { return easySolution }
+
+    if (m & 1) == 0 {
+        // m is even
+        if (n & 1) == 1 {
+            // and n is odd
+            return gcdBinaryRecursiveStein(m >> 1, n)
+        } else {
+            // both m and n are even
+            return gcdBinaryRecursiveStein(m >> 1, n >> 1) << 1
+        }
+    } else if (n & 1) == 0 {
+        // m is odd, n is even
+        return gcdBinaryRecursiveStein(m, n >> 1)
+    } else if (m > n) {
+        // reduce larger argument
+        return gcdBinaryRecursiveStein((m - n) >> 1, n)
+    } else {
+        // reduce larger argument
+        return gcdBinaryRecursiveStein((n - m) >> 1, m)
+    }
+}
+

Depending on your application and your input expectations, it might be reasonable to also search for an "easy solution" using the other gcd implementations:

+
func findEasySolution(_ m: Int, _ n: Int) -> Int? {
+    if m == n {
+        return m
+    }
+    if m == 0 {
+        return n
+    }
+    if n == 0 {
+        return m
+    }
+    return nil
+}
+
+

Put it in a playground and compare it with the results of the other gcd implementations:

+
gcd(52, 39, using: gcdBinaryRecursiveStein)        // 13
+gcd(228, 36, using: gcdBinaryRecursiveStein)       // 12
+gcd(51357, 3819, using: gcdBinaryRecursiveStein)   // 57
+

Least Common Multiple

+

Another algorithm related to the GCD is the least common multiple or LCM.

+

The least common multiple of two numbers a and b is the smallest positive integer that is a multiple of both. In other words, the LCM is evenly divisible by a and b. The example implementation of the LCM takes two numbers and an optional specification which GCD algorithm is used.

+

For example: lcm(2, 3, using: gcdRecursiveEuklid) = 6 , which tells us that 6 is the smallest number that can be devided by 2 as well as 3.

+

We can calculate the LCM using Euklid's algorithm too:

+

a * b
+lcm(a, b) = ---------
+gcd(a, b)

+

In code:

+
func lcm(_ m: Int, _ n: Int, using gcdAlgorithm: (Int, Int) -> (Int) = gcdIterativeEuklid) throws -> Int {
+guard (m & n) != 0 else { throw LCMError.divisionByZero }
+return m / gcdAlgorithm(m, n) * n
+}
+

And to try it out in a playground:

+
lcm(10, 8)    // 40
+

Discussion

+

While these algorithms all calculate the same result, comparing their plane complexity might not be enough to decide for one of them, though. The original iterative Euklidean algorithm is easier to understand. The recursive Euklidean and Stein's algorithm, while being generally faster, their runtime is heavily dependend on the environment they are running on.
+If a fast calculation of the gcd is necessary, a runtime comparison for the specific platform and compiler optimization level should be done for the rekursive Euklidean and Stein's algorithm.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+

Extended by Simon C. Krüger

+ + diff --git a/Genetic/index.html b/Genetic/index.html new file mode 100644 index 000000000..5150af761 --- /dev/null +++ b/Genetic/index.html @@ -0,0 +1,236 @@ + + + Genetic + + + +

Genetic Algorthim

+

What is it?

+

A genetic algorithm (GA) is process inspired by natural selection to find high quality solutions. Most commonly used for optimization. GAs rely on the bio-inspired processes of natural selection, more specifically the process of selection (fitness), crossover and mutation. To understand more, let's walk through these processes in terms of biology:

+

Selection

+
+

Selection, in biology, the preferential survival and reproduction or preferential elimination of individuals with certain genotypes (genetic compositions), by means of natural or artificial controlling factors.

+
+

In other words, survival of the fittest. Organisms that survive in their environment tend to reproduce more. With GAs we generate a fitness model that will rank individuals and give them a better chance for reproduction.

+

Crossover

+
+

Chromosomal crossover (or crossing over) is the exchange of genetic material between homologous chromosomes that results in recombinant chromosomes during sexual reproduction Wikipedia

+
+

Simply reproduction. A generation will be a mixed representation of the previous generation, with offspring taking DNA from both parents. GAs do this by randomly, but weightily, mating offspring to create new generations.

+

Mutation

+
+

Mutation, an alteration in the genetic material (the genome) of a cell of a living organism or of a virus that is more or less permanent and that can be transmitted to the cell’s or the virus’s descendants. Britannica

+
+

The randomization that allows for organisms to change over time. In GAs we build a randomization process that will mutate offspring in a population in order to introduce fitness variance.

+

Resources:

+ +

The Code

+

Problem

+

For this quick and dirty example, we are going to produce an optimized string using a simple genetic algorithm. More specifically we are trying to take a randomly generated origin string of a fixed length and evolve it into the most optimized string of our choosing.

+

We will be creating a bio-inspired world where the absolute existence is the string Hello, World!. Nothing in this universe is better and it's our goal to get as close to it as possible to ensure survival.

+

Define the Universe

+

Before we dive into the core processes we need to set up our "universe". First let's define a lexicon, a set of everything that exists in our universe.

+
let lex: [UInt8] = " !\"#$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~".asciiArray
+

To make things easier, we are actually going to work in Unicode values, so let's define a String extension to help with that.

+
extension String {
+  var unicodeArray: [UInt8] {
+    return [UInt8](self.utf8)
+  }
+}
+

Now, let's define a few global variables for the universe:

+ +
let OPTIMAL:[UInt8] = "Hello, World".unicodeArray
+let DNA_SIZE = OPTIMAL.count
+let POP_SIZE = 50
+let GENERATIONS = 5000
+let MUTATION_CHANCE = 100
+

Population Zero

+

Before selecting, crossover and mutation, we need a population to start with. Now that we have the universe defined we can write that function:

+
func randomPopulation(from lexicon: [UInt8], populationSize: Int, dnaSize: Int) -> [[UInt8]] {
+   guard lexicon.count > 1 else { return [] }
+   var pop = [[UInt8]]()
+
+   (0..<populationSize).forEach { _ in
+       var dna = [UInt8]()
+       (0..<dnaSize).forEach { _ in
+           let char = lexicon.randomElement()! // guaranteed to be non-nil by initial guard statement
+           dna.append(char)
+       }
+       pop.append(dna)
+   }
+   return pop
+}
+

Selection

+

There are two parts to the selection process, the first is calculating the fitness, which will assign a rating to a individual. We do this by simply calculating how close the individual is to the optimal string using unicode values:

+
func calculateFitness(dna: [UInt8], optimal: [UInt8]) -> Int {
+    guard dna.count == optimal.count else { return -1 }
+    var fitness = 0
+    for index in dna.indices {
+        fitness += abs(Int(dna[index]) - Int(optimal[index]))
+    }
+    return fitness
+}
+

The above will produce a fitness value to an individual. The perfect solution, "Hello, World" will have a fitness of 0. "Gello, World" will have a fitness of 1 since it is one unicode value off from the optimal (H->G).

+

This example is very simple, but it'll work for our example. In a real world problem, the optimal solution is unknown or impossible. Here is a paper about optimizing a solution for the famous traveling salesman problem using a GA. In this example the problem is unsolvable by modern computers, but you can rate a individual solution by distance traveled. The optimal fitness here is an impossible 0. The closer the solution is to 0, the better chance for survival. In our example we will reach our goal, a fitness of 0.

+

The second part to selection is weighted choice, also called roulette wheel selection. This defines how individuals are selected for the reproduction process out of the current population. Just because you are the best choice for natural selection doesn't mean the environment will select you. The individual could fall off a cliff, get dysentery or be unable to reproduce.

+

Let's take a second and ask why on this one. Why would you not always want to select the most fit from a population? It's hard to see from this simple example, but let's think about dog breeding, because breeders remove this process and hand select dogs for the next generation. As a result you get improved desired characteristics, but the individuals will also continue to carry genetic disorders that come along with those traits. A certain "branch" of evolution may beat out the current fittest solution at a later time. This may be ok depending on the problem, but to keep this educational we will go with the bio-inspired way.

+

With all that, here is our weight choice function:

+

func weightedChoice(items: [(dna: [UInt8], weight: Double)]) -> (dna: [UInt8], weight: Double) {

+
let total = items.reduce(0) { $0 + $1.weight }
+var n = Double.random(in: 0..<(total * 1000000)) / 1000000.0
+
+for item in items {
+    if n < item.weight {
+        return item
+    }
+    n = n - item.weight
+}
+return items[1]
+
+

}

+

The above function takes a list of individuals with their calculated fitness. Then selects one at random offset by their fitness value. The horrible 1,000,000 multiplication and division is to insure precision by calculating decimals. Double.random only uses integers so this is required to convert to a precise Double, it's not perfect, but enough for our example.

+

Mutation

+

The all powerful mutation, the thing that introduces otherwise non existent fitness variance. It can either hurt of improve a individuals fitness but over time it will cause evolution towards more fit populations. Imagine if our initial random population was missing the charachter H, in that case we need to rely on mutation to introduce that character into the population in order to achieve the optimal solution.

+
func mutate(lexicon: [UInt8], dna: [UInt8], mutationChance: Int) -> [UInt8] {
+    var outputDna = dna
+    (0..<dna.count).forEach { i in
+        let rand = Int.random(in: 0..<mutationChance)
+        if rand == 1 {
+            outputDna[i] = lexicon.randomElement()!
+        }
+    }
+    
+    return outputDna
+}
+

Takes a mutation chance and a individual and returns that individual with mutations, if any.

+

This allows for a population to explore all the possibilities of it's building blocks and randomly stumble on a better solution. If there is too much mutation, the evolution process will get nowhere. If there is too little the populations will become too similar and never be able to branch out of a defect to meet their changing environment.

+

Crossover

+

Crossover, the sexy part of a GA, is how offspring are created from 2 selected individuals in the current population. This is done by splitting the parents into 2 parts, then combining 1 part from each parent to create the offspring. To promote diversity, we randomly select a index to split the parents.

+
func crossover(dna1: [UInt8], dna2: [UInt8], dnaSize: Int) -> [UInt8] {
+    let pos = Int.random(in: 0..<dnaSize)
+    
+    let dna1Index1 = dna1.index(dna1.startIndex, offsetBy: pos)
+    let dna2Index1 = dna2.index(dna2.startIndex, offsetBy: pos)
+    
+    return [UInt8](dna1.prefix(upTo: dna1Index1) + dna2.suffix(from: dna2Index1))
+}
+

The above is used to generate a completely new generation based on the current generation.

+

Putting it all together -- Running the Genetic Algorithm

+

We now have all the functions we need to kick off the algorithm. Let's start from the beginning, first we need a random population to serve as a starting point. We will also initialize a fittest variable to hold the fittest individual, we will initialize it with the first individual of our random population.

+
var population:[[UInt8]] = randomPopulation(from: lex, populationSize: POP_SIZE, dnaSize: DNA_SIZE)
+var fittest = population[0]
+

Now for the meat, the remainder of the code will take place in the generation loop, running once for every generation:

+
for generation in 0...GENERATIONS {
+  // run
+}
+

Now, for each individual in the population, we need to calculate its fitness and weighted value. Since 0 is the best value we will use 1/fitness to represent the weight. Note this is not a percent, but just how much more likely the value is to be selected over others. If the highest number was the most fit, the weight calculation would be fitness/totalFitness, which would be a percent.

+
var weightedPopulation = [(dna:[UInt8], weight:Double)]()
+
+for individual in population {
+  let fitnessValue = calculateFitness(dna: individual, optimal: OPTIMAL)
+  let pair = ( individual, fitnessValue == 0 ? 1.0 : 1.0/Double( fitnessValue ) )
+  weightedPopulation.append(pair)
+}
+

From here we can start to build the next generation.

+
var nextGeneration = []
+

The below loop is where we pull everything together. We loop for POP_SIZE, selecting 2 individuals by weighted choice, crossover their values to produce a offspring, then finial subject the new individual to mutation. Once completed we have a completely new generation based on the last generation.

+
0...POP_SIZE).forEach { _ in
+    let ind1 = weightedChoice(items: weightedPopulation)
+    let ind2 = weightedChoice(items: weightedPopulation)
+
+    let offspring = crossover(dna1: ind1.dna, dna2: ind2.dna, dnaSize: DNA_SIZE)
+
+    // append to the population and mutate
+    nextGeneration.append(mutate(lexicon: lex, dna: offspring, mutationChance: MUTATION_CHANCE))
+}
+

The final piece to the main loop is to select the fittest individual of a population:

+
fittest = population[0]
+var minFitness = calculateFitness(dna: fittest, optimal: OPTIMAL)
+
+population.forEach { indv in
+    let indvFitness = calculateFitness(dna: indv, optimal: OPTIMAL)
+    if indvFitness < minFitness {
+        fittest = indv
+        minFitness = indvFitness
+    }
+}
+if minFitness == 0 { break; }
+print("\(generation): \(String(bytes: fittest, encoding: .utf8)!)")
+

Since we know the fittest string, I've added a break to kill the program if we find it. At the end of a loop add a print statement for the fittest string:

+
print("fittest string: \(String(bytes: fittest, encoding: .utf8)!)")
+

Now we can run the program! Playgrounds are a nice place to develop, but are going to run this program very slow. I highly suggest running in Terminal: swift gen.swift. When running you should see something like this and it should not take too long to get Hello, World:

+
0: RXclh F HDko
+1: DkyssjgElk];
+2: TiM4u) DrKvZ
+3: Dkysu) DrKvZ
+4: -kysu) DrKvZ
+5: Tlwsu) DrKvZ
+6: Tlwsu) Drd}k
+7: Tlwsu) Drd}k
+8: Tlwsu) Drd}k
+9: Tlwsu) Drd}k
+10: G^csu) |zd}k
+11: G^csu) |zdko
+12: G^csu) |zdko
+13: Dkysu) Drd}k
+14: G^wsu) `rd}k
+15: Dkysu) `rdko
+16: Dkysu) `rdko
+17: Glwsu) `rdko
+18: TXysu) `rdkc
+19: U^wsu) `rdko
+20: G^wsu) `rdko
+21: Glysu) `rdko
+22: G^ysu) `rdko
+23: G^ysu) `ryko
+24: G^wsu) `rdko
+25: G^wsu) `rdko
+26: G^wsu) `rdko
+...
+1408: Hello, Wormd
+1409: Hello, Wormd
+1410: Hello, Wormd
+1411: Hello, Wormd
+1412: Hello, Wormd
+1413: Hello, Wormd
+1414: Hello, Wormd
+1415: Hello, Wormd
+1416: Hello, Wormd
+1417: Hello, Wormd
+1418: Hello, Wormd
+1419: Hello, Wormd
+1420: Hello, Wormd
+1421: Hello, Wormd
+1422: Hello, Wormd
+1423: Hello, Wormd
+1424: Hello, Wormd
+1425: Hello, Wormd
+1426: Hello, Wormd
+1427: Hello, Wormd
+1428: Hello, Wormd
+1429: Hello, Wormd
+1430: Hello, Wormd
+1431: Hello, Wormd
+1432: Hello, Wormd
+1433: Hello, Wormd
+1434: Hello, Wormd
+1435: Hello, Wormd
+fittest string: Hello, World
+
+

How long it takes will vary since this is based on randomization, but it should almost always finish in under 5000 generations. Woo!

+

Now What?

+

We did it, we have a running simple genetic algorithm. Take some time a play around with the global variables, POP_SIZE, OPTIMAL, MUTATION_CHANCE, GENERATIONS. Just make sure to only add characters that are in the lexicon or update the lexicon.

+

For an example let's try something much longer: Ray Wenderlich's Swift Algorithm Club Rocks. Plug that string into OPTIMAL and change GENERATIONS to 10000. You'll be able to see that the we are getting somewhere, but you most likely will not reach the optimal string in 10,000 generations. Since we have a larger string let's raise our mutation chance to 200 (1/2 as likely to mutate). You may not get there, but you should get a lot closer than before. With a longer string, too much mutation can make it hard for fit strings to survive. Now try either upping POP_SIZE or increase GENERATIONS. Either way you should eventually get the value, but there will be a "sweet spot" for an string of a certain size.

+

Please submit any kind of update to this tutorial or add more examples!

+ + diff --git a/Graph/index.html b/Graph/index.html new file mode 100644 index 000000000..a80105975 --- /dev/null +++ b/Graph/index.html @@ -0,0 +1,223 @@ + + + Graph + + + +

Graph

+
+

This topic has been tutorialized here

+
+

A graph looks like the following picture:

+

A graph

+

In computer science, a graph is defined as a set of vertices paired with a set of edges. The vertices are represented by circles, and the edges are the lines between them. Edges connect a vertex to other vertices.

+
+

Note: Vertices are sometimes called "nodes", and edges are called "links".

+
+

A graph can represent a social network. Each person is a vertex, and people who know each other are connected by edges. Here is a somewhat historically inaccurate example:

+

Social network

+

Graphs have various shapes and sizes. The edges can have a weight, where a positive or negative numeric value is assigned to each edge. Consider an example of a graph representing airplane flights. Cities can be vertices, and flights can be edges. Then, an edge weight could describe flight time or the price of a ticket.

+

Airplane flights

+

With this hypothetical airline, flying from San Francisco to Moscow is cheapest by going through New York.

+

Edges can also be directed. In examples mentioned above, the edges are undirected. For instance, if Ada knows Charles, then Charles also knows Ada. A directed edge, on the other hand, implies a one-way relationship. A directed edge from vertex X to vertex Y connects X to Y, but not Y to X.

+

Continuing from the flights example, a directed edge from San Francisco to Juneau in Alaska indicates that there is a flight from San Francisco to Juneau, but not from Juneau to San Francisco (I suppose that means you're walking back).

+

One-way flights

+

The following are also graphs:

+

Tree and linked list

+

On the left is a tree structure, on the right a linked list. They can be considered graphs but in a simpler form. They both have vertices (nodes) and edges (links).

+

The first graph includes cycles, where you can start off at a vertex, follow a path, and come back to the original vertex. A tree is a graph without such cycles.

+

Another common type of graph is the directed acyclic graph or DAG:

+

DAG

+

Like a tree, this graph does not have any cycles (no matter where you start, there is no path back to the starting vertex), but this graph has directional edges with the shape that does not necessarily form a hierarchy.

+

Why use graphs?

+

Maybe you're shrugging your shoulders and thinking, what's the big deal? Well, it turns out that a graph is a useful data structure.

+

If you have a programming problem where you can represent your data as vertices and edges, then you can draw your problem as a graph and use well-known graph algorithms such as breadth-first search or depth-first search to find a solution.

+

For example, suppose you have a list of tasks where some tasks have to wait on others before they can begin. You can model this using an acyclic directed graph:

+

Tasks as a graph

+

Each vertex represents a task. An edge between two vertices means the source task must be completed before the destination task can start. As an example, task C cannot start before B and D are finished, and B nor D can start before A is finished.

+

Now that the problem is expressed using a graph, you can use a depth-first search to perform a topological sort. This will put the tasks in an optimal order so that you minimize the time spent waiting for tasks to complete. (One possible order here is A, B, D, E, C, F, G, H, I, J, K.)

+

Whenever you are faced with a tough programming problem, ask yourself, "how can I express this problem using a graph?" Graphs are all about representing relationships between your data. The trick is in how you define "relationships".

+

If you are a musician you might appreciate this graph:

+

Chord map

+

The vertices are chords from the C major scale. The edges -- the relationships between the chords -- represent how likely one chord is to follow another. This is a directed graph, so the direction of the arrows shows how you can go from one chord to the next. It is also a weighted graph, where the weight of the edges -- portrayed here by line thickness -- shows a strong relationship between two chords. As you can see, a G7-chord is very likely to be followed by a C chord, and much less likely by a Am chord.

+

You are probably already using graphs without even knowing it. Your data model is also a graph (from Apple's Core Data documentation):

+

Core Data model

+

Another common graph used by programmers is the state machine, where edges depict the conditions for transitioning between states. Here is a state machine that models my cat:

+

State machine

+

Graphs are awesome. Facebook made a fortune from their social graph. If you are going to learn any data structure, you must choose the graph and the vast collection of standard graph algorithms.

+

Vertices and edges, oh my!

+

In theory, a graph is just a bunch of vertex and edge objects, but how do you describe this in code?

+

There are two main strategies: adjacency list and adjacency matrix.

+

Adjacency List. In an adjacency list implementation, each vertex stores a list of edges that originate from that vertex. For example, if vertex A has an edge to vertices B, C, and D, then vertex A would have a list containing 3 edges.

+

Adjacency list

+

The adjacency list describes outgoing edges. A has an edge to B, but B does not have an edge back to A, so A does not appear in B's adjacency list. Finding an edge or weight between two vertices can be expensive because there is no random access to edges. You must traverse the adjacency lists until it is found.

+

Adjacency Matrix. In an adjacency matrix implementation, a matrix with rows and columns representing vertices stores a weight to indicate if vertices are connected and by what weight. For example, if there is a directed edge of weight 5.6 from vertex A to vertex B, then the entry with row for vertex A and column for vertex B would have the value 5.6:

+

Adjacency matrix

+

Adding another vertex to the graph is expensive, because a new matrix structure must be created with enough space to hold the new row/column, and the existing structure must be copied into the new one.

+

So which one should you use? Most of the time, the adjacency list is the right approach. What follows is a more detailed comparison between the two.

+

Let V be the number of vertices in the graph, and E the number of edges. Then we have:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
OperationAdjacency ListAdjacency Matrix
Storage SpaceO(V + E)O(V^2)
Add VertexO(1)O(V^2)
Add EdgeO(1)O(1)
Check AdjacencyO(V)O(1)
+

"Checking adjacency" means that we try to determine that a given vertex is an immediate neighbor of another vertex. The time to check adjacency for an adjacency list is O(V), because in the worst case a vertex is connected to every other vertex.

+

In the case of a sparse graph, where each vertex is connected to only a handful of other vertices, an adjacency list is the best way to store the edges. If the graph is dense, where each vertex is connected to most of the other vertices, then a matrix is preferred.

+

Here are sample implementations of both adjacency list and adjacency matrix:

+

The code: edges and vertices

+

The adjacency list for each vertex consists of Edge objects:

+
public struct Edge<T>: Equatable where T: Equatable, T: Hashable {
+
+  public let from: Vertex<T>
+  public let to: Vertex<T>
+
+  public let weight: Double?
+
+}
+

This struct describes the "from" and "to" vertices and a weight value. Note that an Edge object is always directed, a one-way connection (shown as arrows in the illustrations above). If you want an undirected connection, you also need to add an Edge object in the opposite direction. Each Edge optionally stores a weight, so they can be used to describe both weighted and unweighted graphs.

+

The Vertex looks like this:

+
public struct Vertex<T>: Equatable where T: Equatable, T: Hashable {
+
+  public var data: T
+  public let index: Int
+
+}
+

It stores arbitrary data with a generic type T, which is Hashable to enforce uniqueness, and also Equatable. Vertices themselves are also Equatable.

+

The code: graphs

+
+

Note: There are many ways to implement graphs. The code given here is just one possible implementation. You probably want to tailor the graph code to each individual problem you are trying to solve. For instance, your edges may not need a weight property, or you may not have the need to distinguish between directed and undirected edges.

+
+

Here is an example of a simple graph:

+

Demo

+

We can represent it as an adjacency matrix or adjacency list. The classes implementing those concepts both inherit a common API from AbstractGraph, so they can be created in an identical fashion, with different optimized data structures behind the scenes.

+

Let's create some directed, weighted graphs, using each representation, to store the example:

+
for graph in [AdjacencyMatrixGraph<Int>(), AdjacencyListGraph<Int>()] {
+
+  let v1 = graph.createVertex(1)
+  let v2 = graph.createVertex(2)
+  let v3 = graph.createVertex(3)
+  let v4 = graph.createVertex(4)
+  let v5 = graph.createVertex(5)
+
+  graph.addDirectedEdge(v1, to: v2, withWeight: 1.0)
+  graph.addDirectedEdge(v2, to: v3, withWeight: 1.0)
+  graph.addDirectedEdge(v3, to: v4, withWeight: 4.5)
+  graph.addDirectedEdge(v4, to: v1, withWeight: 2.8)
+  graph.addDirectedEdge(v2, to: v5, withWeight: 3.2)
+
+}
+

As mentioned earlier, to create an undirected edge you need to make two directed edges. For undirected graphs, we call the following method instead:

+
  graph.addUndirectedEdge(v1, to: v2, withWeight: 1.0)
+  graph.addUndirectedEdge(v2, to: v3, withWeight: 1.0)
+  graph.addUndirectedEdge(v3, to: v4, withWeight: 4.5)
+  graph.addUndirectedEdge(v4, to: v1, withWeight: 2.8)
+  graph.addUndirectedEdge(v2, to: v5, withWeight: 3.2)
+

We could provide nil as the values for the withWeight parameter in either case to make unweighted graphs.

+

The code: adjacency list

+

To maintain the adjacency list, there is a class that maps a list of edges to a vertex. The graph simply maintains an array of such objects and modifies them as necessary.

+
private class EdgeList<T> where T: Equatable, T: Hashable {
+
+  var vertex: Vertex<T>
+  var edges: [Edge<T>]? = nil
+
+  init(vertex: Vertex<T>) {
+    self.vertex = vertex
+  }
+
+  func addEdge(_ edge: Edge<T>) {
+    edges?.append(edge)
+  }
+
+}
+

They are implemented as a class as opposed to structs, so we can modify them by reference, in place, like when adding an edge to a new vertex, where the source vertex already has an edge list:

+
open override func createVertex(_ data: T) -> Vertex<T> {
+  // check if the vertex already exists
+  let matchingVertices = vertices.filter() { vertex in
+    return vertex.data == data
+  }
+
+  if matchingVertices.count > 0 {
+    return matchingVertices.last!
+  }
+
+  // if the vertex doesn't exist, create a new one
+  let vertex = Vertex(data: data, index: adjacencyList.count)
+  adjacencyList.append(EdgeList(vertex: vertex))
+  return vertex
+}
+

The adjacency list for the example looks like this:

+
v1 -> [(v2: 1.0)]
+v2 -> [(v3: 1.0), (v5: 3.2)]
+v3 -> [(v4: 4.5)]
+v4 -> [(v1: 2.8)]
+
+

where the general form a -> [(b: w), ...] means an edge exists from a to b with weight w (with possibly more edges connecting a to other vertices as well).

+

The code: adjacency matrix

+

We will keep track of the adjacency matrix in a two-dimensional [[Double?]] array. An entry of nil indicates no edge, while any other value indicates an edge of the given weight. If adjacencyMatrix[i][j] is not nil, then there is an edge from vertex i to vertex j.

+

To index into the matrix using vertices, we use the index property in Vertex, which is assigned when creating the vertex through the graph object. When creating a new vertex, the graph must resize the matrix:

+
open override func createVertex(_ data: T) -> Vertex<T> {
+  // check if the vertex already exists
+  let matchingVertices = vertices.filter() { vertex in
+    return vertex.data == data
+  }
+
+  if matchingVertices.count > 0 {
+    return matchingVertices.last!
+  }
+
+  // if the vertex doesn't exist, create a new one
+  let vertex = Vertex(data: data, index: adjacencyMatrix.count)
+
+  // Expand each existing row to the right one column.
+  for i in 0 ..< adjacencyMatrix.count {
+    adjacencyMatrix[i].append(nil)
+  }
+
+  // Add one new row at the bottom.
+  let newRow = [Double?](repeating: nil, count: adjacencyMatrix.count + 1)
+  adjacencyMatrix.append(newRow)
+
+  _vertices.append(vertex)
+
+  return vertex
+}
+

Then the adjacency matrix looks like this:

+
[[nil, 1.0, nil, nil, nil]    v1
+ [nil, nil, 1.0, nil, 3.2]    v2
+ [nil, nil, nil, 4.5, nil]    v3
+ [2.8, nil, nil, nil, nil]    v4
+ [nil, nil, nil, nil, nil]]   v5
+
+  v1   v2   v3   v4   v5
+
+

See also

+

This article described what a graph is, and how you can implement the basic data structure. We have other articles on practical uses of graphs, so check those out too!

+

Written by Donald Pinckney and Matthijs Hollemans

+ + diff --git a/Hash Set/index.html b/Hash Set/index.html new file mode 100644 index 000000000..3d9790b93 --- /dev/null +++ b/Hash Set/index.html @@ -0,0 +1,157 @@ + + + Hash Set + + + +

Hash Set

+

A set is a collection of elements that is kind of like an array but with two important differences: the order of the elements in the set is unimportant and each element can appear only once.

+

If the following were arrays, they'd all be different. However, they all represent the same set:

+
[1 ,2, 3]
+[2, 1, 3]
+[3, 2, 1]
+[1, 2, 2, 3, 1]
+

Because each element can appear only once, it doesn't matter how often you write the element down -- only one of them counts.

+
+

Note: I often prefer to use sets over arrays when I have a collection of objects but don't care what order they are in. Using a set communicates to the programmer that the order of the elements is unimportant. If you're using an array, then you can't assume the same thing.

+
+

Typical operations on a set are:

+ +

Union, intersection, and difference are ways to combine two sets into a single one:

+

Union, intersection, difference

+

As of Swift 1.2, the standard library includes a built-in Set type but here I'll show how you can make your own. You wouldn't use this in production code, but it's instructive to see how sets are implemented.

+

It's possible to implement a set using a simple array but that's not the most efficient way. Instead, we'll use a dictionary. Since Swift's dictionary is built using a hash table, our own set will be a hash set.

+

The code

+

Here are the beginnings of HashSet in Swift:

+
public struct HashSet<T: Hashable> {
+    fileprivate var dictionary = Dictionary<T, Bool>()
+
+    public init() {
+
+    }
+
+    public mutating func insert(_ element: T) {
+        dictionary[element] = true
+    }
+
+    public mutating func remove(_ element: T) {
+        dictionary[element] = nil
+    }
+
+    public func contains(_ element: T) -> Bool {
+        return dictionary[element] != nil
+    }
+
+    public func allElements() -> [T] {
+        return Array(dictionary.keys)
+    }
+
+    public var count: Int {
+        return dictionary.count
+    }
+
+    public var isEmpty: Bool {
+        return dictionary.isEmpty
+    }
+}
+

The code is really very simple because we rely on Swift's built-in Dictionary to do all the hard work. The reason we use a dictionary is that dictionary keys must be unique, just like the elements from a set. In addition, a dictionary has O(1) time complexity for most of its operations, making this set implementation very fast.

+

Because we're using a dictionary, the generic type T must conform to Hashable. You can put any type of object into our set, as long as it can be hashed. (This is true for Swift's own Set too.)

+

Normally, you use a dictionary to associate keys with values, but for a set we only care about the keys. That's why we use Bool as the dictionary's value type, even though we only ever set it to true, never to false. (We could have picked anything here but booleans take up the least space.)

+

Copy the code to a playground and add some tests:

+
var set = HashSet<String>()
+
+set.insert("one")
+set.insert("two")
+set.insert("three")
+set.allElements()      // ["one, "three", "two"]
+
+set.insert("two")
+set.allElements()      // still ["one, "three", "two"]
+
+set.contains("one")    // true
+set.remove("one")
+set.contains("one")    // false
+

The allElements() function converts the contents of the set into an array. Note that the order of the elements in that array can be different than the order in which you added the items. As I said, a set doesn't care about the order of the elements (and neither does a dictionary).

+

Combining sets

+

A lot of the usefulness of sets is in how you can combine them. (If you've ever used a vector drawing program like Sketch or Illustrator, you'll have seen the Union, Subtract, Intersect options to combine shapes. Same thing.)

+

Here is the code for the union operation:

+
extension HashSet {
+    public func union(_ otherSet: HashSet<T>) -> HashSet<T> {
+        var combined = HashSet<T>()
+        for obj in self.dictionary.keys {
+            combined.insert(obj)
+        }
+        for obj in otherSet.dictionary.keys {
+            combined.insert(obj)
+        }
+        return combined
+    }
+}
+

The union of two sets creates a new set that consists of all the elements in set A plus all the elements in set B. Of course, if there are duplicate elements they count only once.

+

Example:

+
var setA = HashSet<Int>()
+setA.insert(1)
+setA.insert(2)
+setA.insert(3)
+setA.insert(4)
+
+var setB = HashSet<Int>()
+setB.insert(3)
+setB.insert(4)
+setB.insert(5)
+setB.insert(6)
+
+let union = setA.union(setB)
+union.allElements()           // [5, 6, 2, 3, 1, 4]
+

As you can see, the union of the two sets contains all of the elements now. The values 3 and 4 still appear only once, even though they were in both sets.

+

The intersection of two sets contains only the elements that they have in common. Here is the code:

+
extension HashSet {
+    public func intersect(_ otherSet: HashSet<T>) -> HashSet<T> {
+        var common = HashSet<T>()
+        for obj in dictionary.keys {
+            if otherSet.contains(obj) {
+                common.insert(obj)
+            }
+        }
+        return common
+    }
+}
+

To test it:

+
let intersection = setA.intersect(setB)
+intersection.allElements()
+

This prints [3, 4] because those are the only objects from set A that are also in set B.

+

Finally, the difference between two sets removes the elements they have in common. The code is as follows:

+
extension HashSet {
+    public func difference(_ otherSet: HashSet<T>) -> HashSet<T> {
+        var diff = HashSet<T>()
+        for obj in dictionary.keys {
+            if !otherSet.contains(obj) {
+                diff.insert(obj)
+            }
+        }
+        return diff
+    }
+}
+

It's really the opposite of intersect(). Try it out:

+
let difference1 = setA.difference(setB)
+difference1.allElements()                // [2, 1]
+
+let difference2 = setB.difference(setA)
+difference2.allElements()                // [5, 6]
+

Where to go from here?

+

If you look at the documentation for Swift's own Set, you'll notice it has tons more functionality. An obvious extension would be to make HashSet conform to SequenceType so that you can iterate it with a for...in loop.

+

Another thing you could do is replace the Dictionary with an actual hash table, but one that just stores the keys and doesn't associate them with anything. So you wouldn't need the Bool values anymore.

+

If you often need to look up whether an element belongs to a set and perform unions, then the union-find data structure may be more suitable. It uses a tree structure instead of a dictionary to make the find and union operations very efficient.

+
+

Note: I'd like to make HashSet conform to ArrayLiteralConvertible so you can write let setA: HashSet<Int> = [1, 2, 3, 4] but currently this crashes the compiler.

+
+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Hash Table/index.html b/Hash Table/index.html new file mode 100644 index 000000000..cc6a3ed76 --- /dev/null +++ b/Hash Table/index.html @@ -0,0 +1,184 @@ + + + Hash Table + + + +

Hash Table

+

A hash table allows you to store and retrieve objects by a "key".

+

A hash table is used to implement structures, such as a dictionary, a map, and an associative array. These structures can be implemented by a tree or a plain array, but it is efficient to use a hash table.

+

This should explain why Swift's built-in Dictionary type requires that keys conform to the Hashable protocol: internally it uses a hash table, like the one you will learn about here.

+

How it works

+

A hash table is nothing more than an array. Initially, this array is empty. When you put a value into the hash table under a certain key, it uses that key to calculate an index in the array. Here is an example:

+
hashTable["firstName"] = "Steve"
+
+	The hashTable array:
+	+--------------+
+	| 0:           |
+	+--------------+
+	| 1:           |
+	+--------------+
+	| 2:           |
+	+--------------+
+	| 3: firstName |---> Steve
+	+--------------+
+	| 4:           |
+	+--------------+
+

In this example, the key "firstName" maps to array index 3.

+

Adding a value under a different key puts it at another array index:

+
hashTable["hobbies"] = "Programming Swift"
+
+	The hashTable array:
+	+--------------+
+	| 0:           |
+	+--------------+
+	| 1: hobbies   |---> Programming Swift
+	+--------------+
+	| 2:           |
+	+--------------+
+	| 3: firstName |---> Steve
+	+--------------+
+	| 4:           |
+	+--------------+
+

The trick is how the hash table calculates those array indices. That is where the hashing comes in. When you write the following statement,

+
hashTable["firstName"] = "Steve"
+

the hash table takes the key "firstName" and asks it for its hashValue property. Hence, keys must be Hashable.

+

When you write "firstName".hashValue, it returns a big integer: -4799450059917011053. Likewise, "hobbies".hashValue has the hash value 4799450060928805186. (The values you see may vary.)

+

These numbers are big to be used as indices into our array, and one of them is even negative! A common way to make these big numbers suitable is to first make the hash positive and then take the modulo with the array size.

+

Our array has size 5, so the index for the "firstName" key becomes abs(-4799450059917011053) % 5 = 3. You can calculate that the array index for "hobbies" is 1.

+

Using hashes in this manner is what makes the dictionary efficient: to find an element in the hash table, you must hash the key to get an array index and then look up the element in the underlying array. All these operations take a constant amount of time, so inserting, retrieving, and removing are all O(1).

+
+

Note: It is difficult to predict where in the array your objects end up. Hence, dictionaries do not guarantee any particular order of the elements in the hash table.

+
+

Avoiding collisions

+

There is one problem: because we take the modulo of the hash value with the size of the array, it can happen that two or more keys get assigned the same array index. This is called a collision.

+

One way to avoid collisions is to have a large array which reduces the likelihood of two keys mapping to the same index. Another trick is to use a prime number for the array size. However, collisions are bound to occur, so you need to find a way to handle them.

+

Because our table is small, it is easy to show a collision. For example, the array index for the key "lastName" is also 3, but we do not want to overwrite the value that is already at this array index.

+

A common way to handle collisions is to use chaining. The array looks as follows:

+
	buckets:
+	+-----+
+	|  0  |
+	+-----+     +----------------------------+
+	|  1  |---> | hobbies: Programming Swift |
+	+-----+     +----------------------------+
+	|  2  |
+	+-----+     +------------------+     +----------------+
+	|  3  |---> | firstName: Steve |---> | lastName: Jobs |
+	+-----+     +------------------+     +----------------+
+	|  4  |
+	+-----+
+

With chaining, keys and their values are not stored directly in the array. Instead, each array element is a list of zero or more key/value pairs. The array elements are usually called the buckets and the lists are called the chains. Here we have 5 buckets, and two of these buckets have chains. The other three buckets are empty.

+

If we write the following statement to retrieve an item from the hash table,

+
let x = hashTable["lastName"]
+

it first hashes the key "lastName" to calculate the array index, which is 3. Since bucket 3 has a chain, we step through the list to find the value with the key "lastName". This is done by comparing the keys using a string comparison. The hash table checks that the key belongs to the last item in the chain and returns the corresponding value, "Jobs".

+

Common ways to implement this chaining mechanism are to use a linked list or another array. Since the order of the items in the chain does not matter, you can think of it as a set instead of a list. (Now you can also imagine where the term "bucket" comes from; we just dump all the objects together into the bucket.)

+

Chains should not become long because looking up items in the hash table would become a slow process. Ideally, we would have no chains at all, but in practice it is impossible to avoid collisions. You can improve the odds by giving the hash table enough buckets using high-quality hash functions.

+
+

Note: An alternative to chaining is "open addressing". The idea is this: if an array index is already taken, we put the element in the next unused bucket. This approach has its own upsides and downsides.

+
+

The code

+

Let's look at a basic implementation of a hash table in Swift. We will build it up step-by-step.

+
public struct HashTable<Key: Hashable, Value> {
+  private typealias Element = (key: Key, value: Value)
+  private typealias Bucket = [Element]
+  private var buckets: [Bucket]
+
+  private(set) public var count = 0
+  
+  public var isEmpty: Bool { return count == 0 }
+
+  public init(capacity: Int) {
+    assert(capacity > 0)
+    buckets = Array<Bucket>(repeatElement([], count: capacity))
+  }
+

The HashTable is a generic container, and the two generic types are named Key (which must be Hashable) and Value. We also define two other types: Element is a key/value pair for using in a chain, and Bucket is an array of such Elements.

+

The main array is named buckets. It has a fixed size, the so-called capacity, provided by the init(capacity) method. We are also keeping track of how many items have been added to the hash table using the count variable.

+

An example of how to create a new hash table object:

+
var hashTable = HashTable<String, String>(capacity: 5)
+

The hash table does not do anything yet, so let's add the remaining functionality. First, add a helper method that calculates the array index for a given key:

+
  private func index(forKey key: Key) -> Int {
+    return abs(key.hashValue % buckets.count)
+  }
+

This performs the calculation you saw earlier: it takes the absolute value of the key's hashValue modulo the size of the buckets array. We have put this in a function of its own because it gets used in a few different places.

+

There are four common things you will do with a hash table or dictionary:

+ +

The syntax for these is:

+
hashTable["firstName"] = "Steve"   // insert
+let x = hashTable["firstName"]     // lookup
+hashTable["firstName"] = "Tim"     // update
+hashTable["firstName"] = nil       // delete
+

We can do all these things with a subscript function:

+
  public subscript(key: Key) -> Value? {
+    get {
+      return value(forKey: key)
+    }
+    set {
+      if let value = newValue {
+        updateValue(value, forKey: key)
+      } else {
+        removeValue(forKey: key)
+      }
+    }
+  }
+

This calls three helper functions to do the actual work. Let's take a look at value(forKey:)which retrieves an object from the hash table.

+
  public func value(forKey key: Key) -> Value? {
+    let index = self.index(forKey: key)
+    for element in buckets[index] {
+      if element.key == key {
+        return element.value
+      }
+    }
+    return nil  // key not in hash table
+  }
+

First it calls index(forKey:) to convert the key into an array index. That gives us the bucket number, but this bucket may be used by more than one key if there were collisions. The value(forKey:) loops through the chain from that bucket and compares the keys one-by-one. If found, it returns the corresponding value, otherwise it returns nil.

+

The code to insert a new element or update an existing element lives in updateValue(_:forKey:). This is more complicated:

+
  public mutating func updateValue(_ value: Value, forKey key: Key) -> Value? {
+    let index = self.index(forKey: key)
+    
+    // Do we already have this key in the bucket?
+    for (i, element) in buckets[index].enumerated() {
+      if element.key == key {
+        let oldValue = element.value
+        buckets[index][i].value = value
+        return oldValue
+      }
+    }
+    
+    // This key isn't in the bucket yet; add it to the chain.
+    buckets[index].append((key: key, value: value))
+    count += 1
+    return nil
+  }
+

Again, the first step is to convert the key into an array index to find the bucket. Then we loop through the chain for that bucket. If we find the key in the chain, we must update it with the new value. If the key is not in the chain, we insert the new key/value pair to the end of the chain.

+

As you can see, it is important to keep the chains short (by making the hash table large enough). Otherwise, you spend excessive time in these for...in loops and the performance of the hash table will no longer be O(1) but more like O(n).

+

Removing is similar in that again it loops through the chain:

+
  public mutating func removeValue(forKey key: Key) -> Value? {
+    let index = self.index(forKey: key)
+
+    // Find the element in the bucket's chain and remove it.
+    for (i, element) in buckets[index].enumerated() {
+      if element.key == key {
+        buckets[index].remove(at: i)
+        count -= 1
+        return element.value
+      }
+    }
+    return nil  // key not in hash table
+  }
+

These are the basic functions of the hash table. They all work the same way: convert the key into an array index using its hash value, find the bucket, then loop through that bucket's chain and perform the desired operation.

+

Try this stuff out in a playground. It should work just like a standard Swift Dictionary.

+

Resizing the hash table

+

This version of HashTable always uses an array of a fixed size or capacity. If you have many items to store in the hash table, for the capacity, choose a prime number greater than the maximum number of items.

+

The load factor of a hash table is the percentage of the capacity that is currently used. If there are 3 items in a hash table with 5 buckets, then the load factor is 3/5 = 60%.

+

If the hash table is small, and the chains are long, the load factor can become greater than 1, that is not a good idea.

+

If the load factor becomes high, greater than 75%, you can resize the hash table. Adding the code for this condition is left as an exercise for the reader. Keep in mind that making the buckets array larger will change the array indices that the keys map to! This requires you to insert all the elements again after resizing the array.

+

Where to go from here?

+

HashTable is quite basic. It might be efficient to integrate it with the Swift standard library by making it a SequenceType.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Hashed Heap/index.html b/Hashed Heap/index.html new file mode 100644 index 000000000..c9c5ef33c --- /dev/null +++ b/Hashed Heap/index.html @@ -0,0 +1,15 @@ + + + Hashed Heap + + + +

Hashed Heap

+

A hashed heap is a heap with a hash map (also known as a dictionary) to speed up lookup of elements by value. This combination doesn't compromize on time performance but requires extra storage for the hash map. This is mainly used for heuristic search algorihms, in particular A*.

+

The code

+

See HashedHeap.swift for the implementation. See Heap for a detailed explanation of the basic heap implementation.

+

See also

+

Heap on Wikipedia

+

Written for the Swift Algorithm Club by Alejandro Isaza

+ + diff --git a/HaversineDistance/index.html b/HaversineDistance/index.html new file mode 100644 index 000000000..fbbe6f83b --- /dev/null +++ b/HaversineDistance/index.html @@ -0,0 +1,28 @@ + + + HaversineDistance + + + +

Haversine Distance

+

Calculates the distance on a sphere between two points given in latitude and longitude using the haversine formula.

+

The haversine formula can be found on Wikipedia

+

The Haversine Distance is implemented as a function as a class would be kind of overkill.

+

haversineDinstance(la1: Double, lo1: Double, la2: Double, lo2: Double, radius: Double = 6367444.7) -> Double

+ +

The function contains 3 closures in order to make the code more readable and comparable to the Haversine formula given by the Wikipedia page mentioned above.

+
    +
  1. haversine implements the haversine, a trigonometric function.
    2. +
  2. ahaversine the inverse function of the haversine.
    3. +
  3. dToR a closure converting degrees to radians.
    4. +
+

The result of haversineDistance is returned in meters.

+

Written for Swift Algorithm Club by Jaap Wijnen.

+ + diff --git a/Heap Sort/index.html b/Heap Sort/index.html new file mode 100644 index 000000000..95ebd2d56 --- /dev/null +++ b/Heap Sort/index.html @@ -0,0 +1,61 @@ + + + Heap Sort + + + +

Heap Sort

+

Sorts an array from low to high using a heap.

+

A heap is a partially sorted binary tree that is stored inside an array. The heap sort algorithm takes advantage of the structure of the heap to perform a fast sort.

+

To sort from lowest to highest, heap sort first converts the unsorted array to a max-heap, so that the first element in the array is the largest.

+

Let's say the array to sort is:

+
[ 5, 13, 2, 25, 7, 17, 20, 8, 4 ]
+
+

This is first turned into a max-heap that looks like this:

+

The max-heap

+

The heap's internal array is then:

+
[ 25, 13, 20, 8, 7, 17, 2, 5, 4 ]
+
+

That's hardly what you'd call sorted! But now the sorting process starts: we swap the first element (index 0) with the last one at index n-1, to get:

+
[ 4, 13, 20, 8, 7, 17, 2, 5, 25 ]
+  *                          *
+
+

Now the new root node, 4, will be smaller than its children, so we fix up the max-heap up to element n-2 using the shift down or "heapify" procedure. After repairing the heap, the new root is now the second-largest item in the array:

+
[20, 13, 17, 8, 7, 4, 2, 5 | 25]
+
+

Important: When we fix the heap, we ignore the last item at index n-1. That now contains the array's maximum value, so it is in its final sorted place already. The | bar indicates where the sorted portion of the array begins. We'll leave that part of the array alone from now on.

+

Again, we swap the first element with the last one (this time at index n-2):

+
[5, 13, 17, 8, 7, 4, 2, 20 | 25]
+ *                      *
+
+

And fix up the heap to make it valid max-heap again:

+
[17, 13, 5, 8, 7, 4, 2 | 20, 25]
+
+

As you can see, the largest items are making their way to the back. We repeat this process until we arrive at the root node and then the whole array is sorted.

+
+

Note: This process is very similar to selection sort, which repeatedly looks for the minimum item in the remainder of the array. Extracting the minimum or maximum value is what heaps are good at.

+
+

Performance of heap sort is O(n lg n) in best, worst, and average case. Because we modify the array directly, heap sort can be performed in-place. But it is not a stable sort: the relative order of identical elements is not preserved.

+

Here's how you can implement heap sort in Swift:

+
extension Heap {
+  public mutating func sort() -> [T] {
+    for i in stride(from: (elements.count - 1), through: 1, by: -1) {
+      swap(&elements[0], &elements[i])
+      shiftDown(0, heapSize: i)
+    }
+    return elements
+  }
+}
+

This adds a sort() function to our heap implementation. To use it, you would do something like this:

+
var h1 = Heap(array: [5, 13, 2, 25, 7, 17, 20, 8, 4], sort: >)
+let a1 = h1.sort()
+

Because we need a max-heap to sort from low-to-high, you need to give Heap the reverse of the sort function. To sort <, the Heap object must be created with > as the sort function. In other words, sorting from low-to-high creates a max-heap and turns it into a min-heap.

+

We can write a handy helper function for that:

+
public func heapsort<T>(_ a: [T], _ sort: @escaping (T, T) -> Bool) -> [T] {
+  let reverseOrder = { i1, i2 in sort(i2, i1) }
+  var h = Heap(array: a, sort: reverseOrder)
+  return h.sort()
+}
+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Heap/index.html b/Heap/index.html new file mode 100644 index 000000000..f38d426d5 --- /dev/null +++ b/Heap/index.html @@ -0,0 +1,270 @@ + + + Heap + + + +

Heap

+
+

This topic has been tutorialized here

+
+

A heap is a binary tree inside an array, so it does not use parent/child pointers. A heap is sorted based on the "heap property" that determines the order of the nodes in the tree.

+

Common uses for heap:

+ +

The heap property

+

There are two kinds of heaps: a max-heap and a min-heap which are different by the order in which they store the tree nodes.

+

In a max-heap, parent nodes have a greater value than each of their children. In a min-heap, every parent node has a smaller value than its child nodes. This is called the "heap property", and it is true for every single node in the tree.

+

An example:

+

A max-heap

+

This is a max-heap because every parent node is greater than its children. (10) is greater than (7) and (2). (7) is greater than (5) and (1).

+

As a result of this heap property, a max-heap always stores its largest item at the root of the tree. For a min-heap, the root is always the smallest item in the tree. The heap property is useful because heaps are often used as a priority queue to access the "most important" element quickly.

+
+

Note: The root of the heap has the maximum or minimum element, but the sort order of other elements are not predictable. For example, the maximum element is always at index 0 in a max-heap, but the minimum element isn’t necessarily the last one. -- the only guarantee you have is that it is one of the leaf nodes, but not which one.

+
+

How does a heap compare to regular trees?

+

A heap is not a replacement for a binary search tree, and there are similarities and differences between them. Here are some main differences:

+

Order of the nodes. In a binary search tree (BST), the left child must be smaller than its parent, and the right child must be greater. This is not true for a heap. In a max-heap both children must be smaller than the parent, while in a min-heap they both must be greater.

+

Memory. Traditional trees take up more memory than just the data they store. You need to allocate additional storage for the node objects and pointers to the left/right child nodes. A heap only uses a plain array for storage and uses no pointers.

+

Balancing. A binary search tree must be "balanced" so that most operations have O(log n) performance. You can either insert and delete your data in a random order or use something like an AVL tree or red-black tree, but with heaps we don't actually need the entire tree to be sorted. We just want the heap property to be fulfilled, so balancing isn't an issue. Because of the way the heap is structured, heaps can guarantee O(log n) performance.

+

Searching. Whereas searching is fast in a binary tree, it is slow in a heap. Searching isn't a top priority in a heap since the purpose of a heap is to put the largest (or smallest) node at the front and to allow relatively fast inserts and deletes.

+

The tree inside an array

+

An array may seem like an odd way to implement a tree-like structure, but it is efficient in both time and space.

+

This is how we are going to store the tree from the above example:

+
[ 10, 7, 2, 5, 1 ]
+
+

That's all there is to it! We don't need any more storage than just this simple array.

+

So how do we know which nodes are the parents and which are the children if we are not allowed to use any pointers? Good question! There is a well-defined relationship between the array index of a tree node and the array indices of its parent and children.

+

If i is the index of a node, then the following formulas give the array indices of its parent and child nodes:

+
parent(i) = floor((i - 1)/2)
+left(i)   = 2i + 1
+right(i)  = 2i + 2
+
+

Note that right(i) is simply left(i) + 1. The left and right nodes are always stored right next to each other.

+

Let's use these formulas on the example. Fill in the array index and we should get the positions of the parent and child nodes in the array:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NodeArray index (i)Parent indexLeft childRight child
100-112
71034
22056
53178
141910
+

Verify for yourself that these array indices indeed correspond to the picture of the tree.

+
+

Note: The root node (10) does not have a parent because -1 is not a valid array index. Likewise, nodes (2), (5), and (1) do not have children because those indices are greater than the array size, so we always have to make sure the indices we calculate are actually valid before we use them.

+
+

Recall that in a max-heap, the parent's value is always greater than (or equal to) the values of its children. This means the following must be true for all array indices i:

+
array[parent(i)] >= array[i]
+

Verify that this heap property holds for the array from the example heap.

+

As you can see, these equations allow us to find the parent or child index for any node without the need for pointers. It is complicated than just dereferencing a pointer, but that is the tradeoff: we save memory space but pay with extra computations. Fortunately, the computations are fast and only take O(1) time.

+

It is important to understand this relationship between array index and position in the tree. Here is a larger heap which has 15 nodes divided over four levels:

+

Large heap

+

The numbers in this picture are not the values of the nodes but the array indices that store the nodes! Here is the array indices correspond to the different levels of the tree:

+

The heap array

+

For the formulas to work, parent nodes must appear before child nodes in the array. You can see that in the above picture.

+

Note that this scheme has limitations. You can do the following with a regular binary tree but not with a heap:

+

Impossible with a heap

+

You can not start a new level unless the current lowest level is completely full, so heaps always have this kind of shape:

+

The shape of a heap

+
+

Note: You could emulate a regular binary tree with a heap, but it would be a waste of space, and you would need to mark array indices as being empty.

+
+

Pop quiz! Let's say we have the array:

+
[ 10, 14, 25, 33, 81, 82, 99 ]
+
+

Is this a valid heap? The answer is yes! A sorted array from low-to-high is a valid min-heap. We can draw this heap as follows:

+

A sorted array is a valid heap

+

The heap property holds for each node because a parent is always smaller than its children. (Verify for yourself that an array sorted from high-to-low is always a valid max-heap.)

+
+

Note: But not every min-heap is necessarily a sorted array! It only works one way. To turn a heap back into a sorted array, you need to use heap sort.

+
+

More math!

+

In case you are curious, here are a few more formulas that describe certain properties of a heap. You do not need to know these by heart, but they come in handy sometimes. Feel free to skip this section!

+

The height of a tree is defined as the number of steps it takes to go from the root node to the lowest leaf node, or more formally: the height is the maximum number of edges between the nodes. A heap of height h has h + 1 levels.

+

This heap has height 3, so it has 4 levels:

+

Large heap

+

A heap with n nodes has height h = floor(log2(n)). This is because we always fill up the lowest level completely before we add a new level. The example has 15 nodes, so the height is floor(log2(15)) = floor(3.91) = 3.

+

If the lowest level is completely full, then that level contains 2^h nodes. The rest of the tree above it contains 2^h - 1 nodes. Fill in the numbers from the example: the lowest level has 8 nodes, which indeed is 2^3 = 8. The first three levels contain a total of 7 nodes, i.e. 2^3 - 1 = 8 - 1 = 7.

+

The total number of nodes n in the entire heap is therefore 2^(h+1) - 1. In the example, 2^4 - 1 = 16 - 1 = 15.

+

There are at most ceil(n/2^(h+1)) nodes of height h in an n-element heap.

+

The leaf nodes are always located at array indices floor(n/2) to n-1. We will make use of this fact to quickly build up the heap from an array. Verify this for the example if you don't believe it. ;-)

+

Just a few math facts to brighten your day.

+

What can you do with a heap?

+

There are two primitive operations necessary to make sure the heap is a valid max-heap or min-heap after you insert or remove an element:

+ +

Shifting up or down is a recursive procedure that takes O(log n) time.

+

Here are other operations that are built on primitive operations:

+ +

All of the above take time O(log n) because shifting up or down is expensive. There are also a few operations that take more time:

+ +

The heap also has a peek() function that returns the maximum (max-heap) or minimum (min-heap) element, without removing it from the heap. Time: O(1).

+
+

Note: By far the most common things you will do with a heap are inserting new values with insert() and removing the maximum or minimum value with remove(). Both take O(log n) time. The other operations exist to support more advanced usage, such as building a priority queue where the "importance" of items can change after they have been added to the queue.

+
+

Inserting into the heap

+

Let's go through an example of insertion to see in details how this works. We will insert the value 16 into this heap:

+

The heap before insertion

+

The array for this heap is [ 10, 7, 2, 5, 1 ].

+

The first step when inserting a new item is to append it to the end of the array. The array becomes:

+
[ 10, 7, 2, 5, 1, 16 ]
+
+

This corresponds to the following tree:

+

The heap before insertion

+

The (16) was added to the first available space on the last row.

+

Unfortunately, the heap property is no longer satisfied because (2) is above (16), and we want higher numbers above lower numbers. (This is a max-heap.)

+

To restore the heap property, we swap (16) and (2).

+

The heap before insertion

+

We are not done yet because (10) is also smaller than (16). We keep swapping our inserted value with its parent, until the parent is larger or we reach the top of the tree. This is called shift-up or sifting and is done after every insertion. It makes a number that is too large or too small "float up" the tree.

+

Finally, we get:

+

The heap before insertion

+

And now every parent is greater than its children again.

+

The time required for shifting up is proportional to the height of the tree, so it takes O(log n) time. (The time it takes to append the node to the end of the array is only O(1), so that does not slow it down.)

+

Removing the root

+

Let's remove (10) from this tree:

+

The heap before removal

+

What happens to the empty spot at the top?

+

The root is gone

+

When inserting, we put the new value at the end of the array. Here, we do the opposite: we take the last object we have, stick it up on top of the tree, and restore the heap property.

+

The last node goes to the root

+

Let's look at how to shift-down (1). To maintain the heap property for this max-heap, we want to the highest number of top. We have two candidates for swapping places with: (7) and (2). We choose the highest number between these three nodes to be on top. That is (7), so swapping (1) and (7) gives us the following tree.

+

The last node goes to the root

+

Keep shifting down until the node does not have any children or it is larger than both its children. For our heap, we only need one more swap to restore the heap property:

+

The last node goes to the root

+

The time required for shifting all the way down is proportional to the height of the tree which takes O(log n) time.

+
+

Note: shiftUp() and shiftDown() can only fix one out-of-place element at a time. If there are multiple elements in the wrong place, you need to call these functions once for each of those elements.

+
+

Removing any node

+

The vast majority of the time you will be removing the object at the root of the heap because that is what heaps are designed for.

+

However, it can be useful to remove an arbitrary element. This is a general version of remove() and may involve either shiftDown() or shiftUp().

+

Let's take the example tree again and remove (7):

+

The heap before removal

+

As a reminder, the array is:

+
[ 10, 7, 2, 5, 1 ]
+
+

As you know, removing an element could potentially invalidate the max-heap or min-heap property. To fix this, we swap the node that we are removing with the last element:

+
[ 10, 1, 2, 5, 7 ]
+
+

The last element is the one that we will return; we will call removeLast() to remove it from the heap. The (1) is now out-of-order because it is smaller than its child, (5) but sits higher in the tree. We call shiftDown() to repair this.

+

However, shifting down is not the only situation we need to handle. It may also happen that the new element must be shifted up. Consider what happens if you remove (5) from the following heap:

+

We need to shift up

+

Now (5) gets swapped with (8). Because (8) is larger than its parent, we need to call shiftUp().

+

Creating a heap from an array

+

It can be convenient to convert an array into a heap. This just shuffles the array elements around until the heap property is satisfied.

+

In code it would look like this:

+
  private mutating func buildHeap(fromArray array: [T]) {
+    for value in array {
+      insert(value)
+    }
+  }
+

We simply call insert() for each of the values in the array. Simple enough but not very efficient. This takes O(n log n) time in total because there are n elements and each insertion takes log n time.

+

If you didn't gloss over the math section, you'd have seen that for any heap the elements at array indices n/2 to n-1 are the leaves of the tree. We can simply skip those leaves. We only have to process the other nodes, since they are parents with one or more children and therefore may be in the wrong order.

+

In code:

+
  private mutating func buildHeap(fromArray array: [T]) {
+    elements = array
+    for i in stride(from: (nodes.count/2-1), through: 0, by: -1) {
+      shiftDown(index: i, heapSize: elements.count)
+    }
+  }
+

Here, elements is the heap's own array. We walk backwards through this array, starting at the first non-leaf node, and call shiftDown(). This simple loop puts these nodes, as well as the leaves that we skipped, in the correct order. This is known as Floyd's algorithm and only takes O(n) time. Win!

+

Searching the heap

+

Heaps are not made for fast searches, but if you want to remove an arbitrary element using removeAtIndex() or change the value of an element with replace(), then you need to obtain the index of that element. Searching is one way to do this, but it is slow.

+

In a binary search tree, depending on the order of the nodes, a fast search can be guaranteed. Since a heap orders its nodes differently, a binary search will not work, and you need to check every node in the tree.

+

Let's take our example heap again:

+

The heap

+

If we want to search for the index of node (1), we could just step through the array [ 10, 7, 2, 5, 1 ] with a linear search.

+

Even though the heap property was not conceived with searching in mind, we can still take advantage of it. We know that in a max-heap a parent node is always larger than its children, so we can ignore those children (and their children, and so on...) if the parent is already smaller than the value we are looking for.

+

Let's say we want to see if the heap contains the value 8 (it doesn't). We start at the root (10). This is not what we are looking for, so we recursively look at its left and right child. The left child is (7). That is also not what we want, but since this is a max-heap, we know there is no point in looking at the children of (7). They will always be smaller than 7 and are therefore never equal to 8; likewise, for the right child, (2).

+

Despite this small optimization, searching is still an O(n) operation.

+
+

Note: There is a way to turn lookups into a O(1) operation by keeping an additional dictionary that maps node values to indices. This may be worth doing if you often need to call replace() to change the "priority" of objects in a priority queue that's built on a heap.

+
+

The code

+

See Heap.swift for the implementation of these concepts in Swift. Most of the code is quite straightforward. The only tricky bits are in shiftUp() and shiftDown().

+

You have seen that there are two types of heaps: a max-heap and a min-heap. The only difference between them is in how they order their nodes: largest value first or smallest value first.

+

Rather than create two different versions, MaxHeap and MinHeap, there is just one Heap object and it takes an isOrderedBefore closure. This closure contains the logic that determines the order of two values. You have probably seen this before because it is also how Swift's sort() works.

+

To make a max-heap of integers, you write:

+
var maxHeap = Heap<Int>(sort: >)
+

And to create a min-heap you write:

+
var minHeap = Heap<Int>(sort: <)
+

I just wanted to point this out, because where most heap implementations use the < and > operators to compare values, this one uses the isOrderedBefore() closure.

+

See also

+

Heap on Wikipedia

+

Written for the Swift Algorithm Club by Kevin Randrup and Matthijs Hollemans

+ + diff --git a/Huffman Coding/index.html b/Huffman Coding/index.html new file mode 100644 index 000000000..47b1b88dc --- /dev/null +++ b/Huffman Coding/index.html @@ -0,0 +1,326 @@ + + + Huffman Coding + + + +

Huffman Coding

+

The idea: To encode objects that occur often with a smaller number of bits than objects that occur less frequently.

+

Although any type of objects can be encoded with this scheme, it is common to compress a stream of bytes. Suppose you have the following text, where each character is one byte:

+
so much words wow many compression
+
+

If you count how often each byte appears, you can see some bytes occur more than others:

+
space: 5                  u: 1
+    o: 5                  h: 1
+    s: 4                  d: 1
+    m: 3                  a: 1
+    w: 3                  y: 1
+    c: 2                  p: 1
+    r: 2                  e: 1
+    n: 2                  i: 1
+
+

We can assign bit strings to each of these bytes. The more common a byte is, the fewer bits we assign to it. We might get something like this:

+
space: 5    010           u: 1    11001
+    o: 5    000	          h: 1    10001
+    s: 4    101	          d: 1    11010
+    m: 3    111	          a: 1    11011
+    w: 3    0010          y: 1    01111
+    c: 2    0011          p: 1    11000
+    r: 2    1001          e: 1    01110
+    n: 2    0110          i: 1    10000
+
+

Now if we replace the original bytes with these bit strings, the compressed output becomes:

+
101 000 010 111 11001 0011 10001 010 0010 000 1001 11010 101
+s   o   _   m   u     c    h     _   w    o   r    d     s
+
+010 0010 000 0010 010 111 11011 0110 01111 010 0011 000 111
+_   w    o   w    _   m   a     n    y     _   c    o   m
+
+11000 1001 01110 101 101 10000 000 0110 0
+p     r    e     s   s   i     o   n
+
+

The extra 0-bit at the end is there to make a full number of bytes. We were able to compress the original 34 bytes into merely 16 bytes, a space savings of over 50%!

+

To be able to decode these bits, we need to have the original frequency table. That table needs to be transmitted or saved along with the compressed data. Otherwise, the decoder does not know how to interpret the bits. Because of the overhead of this frequency table (about 1 kilobyte), it is not beneficial to use Huffman encoding on small inputs.

+

How it works

+

When compressing a stream of bytes, the algorithm first creates a frequency table that counts how often each byte occurs. Based on this table, the algorithm creates a binary tree that describes the bit strings for each of the input bytes.

+

For our example, the tree looks like this:

+

The compression tree

+

Note that the tree has 16 leaf nodes (the grey ones), one for each byte value from the input. Each leaf node also shows the count of how often it occurs. The other nodes are "intermediate" nodes. The number shown in these nodes is the sum of the counts of their child nodes. The count of the root node is therefore the total number of bytes in the input.

+

The edges between the nodes are either "1" or "0". These correspond to the bit-encodings of the leaf nodes. Notice how each left branch is always 1 and each right branch is always 0.

+

Compression is then a matter of looping through the input bytes and for each byte traversing the tree from the root node to that byte's leaf node. Every time we take a left branch, we emit a 1-bit. When we take a right branch, we emit a 0-bit.

+

For example, to go from the root node to c, we go right (0), right again (0), left (1), and left again (1). This gives the Huffman code as 0011 for c.

+

Decompression works in exactly the opposite way. It reads the compressed bits one-by-one and traverses the tree until it reaches to a leaf node. The value of that leaf node is the uncompressed byte. For example, if the bits are 11010, we start at the root and go left, left again, right, left, and a final right to end up at d.

+

The code

+

Before we get to the actual Huffman coding scheme, it is useful to have some helper code that can write individual bits to an NSData object. The smallest piece of data that NSData understands is the byte, but we are dealing in bits, so we need to translate between the two.

+
public class BitWriter {
+  public var data = NSMutableData()
+  var outByte: UInt8 = 0
+  var outCount = 0
+
+  public func writeBit(bit: Bool) {
+    if outCount == 8 {
+      data.append(&outByte, length: 1)
+      outCount = 0
+    }
+    outByte = (outByte << 1) | (bit ? 1 : 0)
+    outCount += 1
+  }
+
+  public func flush() {
+    if outCount > 0 {
+      if outCount < 8 {
+        let diff = UInt8(8 - outCount)
+        outByte <<= diff
+      }
+      data.append(&outByte, length: 1)
+    }
+  }
+}
+

To add a bit to the NSData, you can call writeBit(). This helper object stuffs each new bit into the outByte variable. Once you have written 8 bits, outByte gets added to the NSData object for real.

+

The flush() method is used for outputting the very last byte. There is no guarantee that the number of compressed bits is a nice round multiple of 8, in which case there may be some spare bits at the end. If so, flush() adds a few 0-bits to make sure that we write a full byte.

+

Here is a similar helper object for reading individual bits from NSData:

+
public class BitReader {
+  var ptr: UnsafePointer<UInt8>
+  var inByte: UInt8 = 0
+  var inCount = 8
+
+  public init(data: NSData) {
+    ptr = data.bytes.assumingMemoryBound(to: UInt8.self)
+  }
+
+  public func readBit() -> Bool {
+    if inCount == 8 {
+      inByte = ptr.pointee    // load the next byte
+      inCount = 0
+      ptr = ptr.successor()
+    }
+    let bit = inByte & 0x80  // read the next bit
+    inByte <<= 1
+    inCount += 1
+    return bit == 0 ? false : true
+  }
+}
+

By using this helper object, we can read one whole byte from the NSData object and put it in inByte. Then, readBit() returns the individual bits from that byte. Once readBit() has been called 8 times, we read the next byte from the NSData.

+
+

Note: If you are unfamiliar with this type of bit manipulation, just know that these two helper objects make it simple for us to write and read bits.

+
+

The frequency table

+

The first step in the Huffman compression is to read the entire input stream and build a frequency table. This table contains a list of all 256 possible byte values and shows how often each of these bytes occurs in the input data.

+

We could store this frequency information in a dictionary or an array, but since we need to build a tree, we might store the frequency table as the leaves of the tree.

+

Here are the definitions we need:

+
class Huffman {
+  typealias NodeIndex = Int
+
+  struct Node {
+    var count = 0
+    var index: NodeIndex = -1
+    var parent: NodeIndex = -1
+    var left: NodeIndex = -1
+    var right: NodeIndex = -1
+  }
+
+  var tree = [Node](repeating: Node(), count: 256)
+
+  var root: NodeIndex = -1
+}
+

The tree structure is stored in the tree array and will be made up of Node objects. Since this is a binary tree, each node needs two children, left and right, and a reference back to its parent node. Unlike a typical binary tree, these nodes do not use pointers to refer to each other but use simple integer indices in the tree array. (We also store the array index of the node itself; the reason for this will become clear later.)

+

Note that the tree currently has room for 256 entries. These are for the leaf nodes because there are 256 possible byte values. Of course, not all of those may end up being used, depending on the input data. Later, we will add more nodes as we build up the actual tree. For the moment, there is not a tree yet. It includes 256 separate leaf nodes with no connections between them. All the node counts are 0.

+

We use the following method to count how often each byte occurs in the input data:

+
  fileprivate func countByteFrequency(inData data: NSData) {
+    var ptr = data.bytes.assumingMemoryBound(to: UInt8.self)
+    for _ in 0..<data.length {
+      let i = Int(ptr.pointee)
+      tree[i].count += 1
+      tree[i].index = i
+      ptr = ptr.successor()
+    }
+  }
+

This steps through the NSData object from beginning to end and for each byte increments the count of the corresponding leaf node. After countByteFrequency() completes, the first 256 Node objects in the tree array represent the frequency table.

+

To decompress a Huffman-encoded block of data, we need to have the original frequency table. If we were writing the compressed data to a file, then somewhere in the file we should include the frequency table.

+

We could dump the first 256 elements from the tree array, but that is not efficient. Not all of these 256 elements will be used, and we do not need to serialize the parent, right, and left pointers. All we need is the frequency information and not the entire tree.

+

Instead, we will add a method to export the frequency table without all the pieces we do not need:

+
  struct Freq {
+    var byte: UInt8 = 0
+    var count = 0
+  }
+
+  func frequencyTable() -> [Freq] {
+    var a = [Freq]()
+    for i in 0..<256 where tree[i].count > 0 {
+      a.append(Freq(byte: UInt8(i), count: tree[i].count))
+    }
+    return a
+  }
+

The frequencyTable() method looks at those first 256 nodes from the tree but keeps only those that are used, without the parent, left, and right pointers. It returns an array of Freq objects. You have to serialize this array along with the compressed data, so that it can be properly decompressed later.

+

The tree

+

As a reminder, there is the compression tree for the example:

+

The compression tree

+

The leaf nodes represent the actual bytes that are present in the input data. The intermediary nodes connect the leaves in such a way that the path from the root to a frequently-used byte value is shorter than the path to a less common byte value. As you can see, m, s, space, and o are the most common letters in our input data and the highest up in the tree.

+

To build the tree, we do the following:

+
    +
  1. Find the two nodes with the smallest counts that do not have a parent node yet.
    2. +
  2. Create a new parent node that links these two nodes together.
    3. +
  3. This repeats over and over until only one node with no parent remains. This becomes the root node of the tree.
    4. +
+

This is an ideal place to use a priority queue. A priority queue is a data structure that is optimized, so that finding the minimum value is always fast. Here, we repeatedly need to find the node with the smallest count.

+

The function buildTree() then becomes:

+
  fileprivate func buildTree() {
+    var queue = PriorityQueue<Node>(sort: { $0.count < $1.count })
+    for node in tree where node.count > 0 {
+      queue.enqueue(node)                            // 1
+    }
+
+    while queue.count > 1 {
+      let node1 = queue.dequeue()!                   // 2
+      let node2 = queue.dequeue()!
+
+      var parentNode = Node()                        // 3
+      parentNode.count = node1.count + node2.count
+      parentNode.left = node1.index
+      parentNode.right = node2.index
+      parentNode.index = tree.count
+      tree.append(parentNode)
+
+      tree[node1.index].parent = parentNode.index    // 4
+      tree[node2.index].parent = parentNode.index
+
+      queue.enqueue(parentNode)                      // 5
+    }
+
+    let rootNode = queue.dequeue()!                  // 6
+    root = rootNode.index
+  }
+

Here is how it works step-by-step:

+
    +
  1. +

    Create a priority queue and enqueue all the leaf nodes that have at least a count of 1. (If the count is 0, then this byte value did not appear in the input data.) The PriorityQueue object sorts the nodes by their count, so that the node with the lowest count is always the first one that gets dequeued.

    +
    2. +
  2. +

    While there are at least two nodes left in the queue, remove the two nodes that are at the front of the queue. Since this is a min-priority queue, this gives us the two nodes with the smallest counts that do not have a parent node yet.

    +
    3. +
  3. +

    Create a new intermediate node that connects node1 and node2. The count of this new node is the sum of the counts of node1 and node2. Because the nodes are connected using array indices instead of real pointers, we use node1.index and node2.index to find these nodes in the tree array. (This is why a Node needs to know its own index.)

    +
    4. +
  4. +

    Link the two nodes into their new parent node. Now this new intermediate node has become part of the tree.

    +
    5. +
  5. +

    Put the new intermediate node back into the queue. At this point we are done with node1 and node2, but the parentNode still needs to be connected to other nodes in the tree.

    +
    6. +
  6. +

    Repeat steps 2-5 until there is only one node left in the queue. This becomes the root node of the tree, and we are done.

    +
    7. +
+

The animation shows what the process looks like:

+

Building the tree

+
+

Note: Instead of using a priority queue, you can repeatedly iterate through the tree array to find the next two smallest nodes, but that makes the compressor slow as O(n^2). Using the priority queue, the running time is only O(n log n) where n is the number of nodes.

+
+
+

Fun fact: Due to the nature of binary trees, if we have x leaf nodes we can at most add x - 1 additional nodes to the tree. Given that at most there will be 256 leaf nodes, the tree will never contain more than 511 nodes total.

+
+

Compression

+

Now that we know how to build the compression tree from the frequency table, we can use it to compress the contents of an NSData object. Here is the code:

+
  public func compressData(data: NSData) -> NSData {
+    countByteFrequency(inData: data)
+    buildTree()
+
+    let writer = BitWriter()
+    var ptr = data.bytes.assumingMemoryBound(to: UInt8.self)
+    for _ in 0..<data.length {
+      let c = ptr.pointee
+      let i = Int(c)
+      traverseTree(writer: writer, nodeIndex: i, childIndex: -1)
+      ptr = ptr.successor()
+    }
+    writer.flush()
+    return writer.data
+  }
+

This first calls countByteFrequency() to build the frequency table and then calls buildTree() to put together the compression tree. It also creates a BitWriter object for writing individual bits.

+

Then, it loops through the entire input and calls traverseTree()for each byte. This method will step through the tree nodes and for each node write a 1 or 0 bit. Finally, we return the BitWriter's data object.

+
+

Note: Compression always requires two passes through the entire input data: first to build the frequency table, and second to convert the bytes to their compressed bit sequences.

+
+

The interesting stuff happens in traverseTree(). This is a recursive method:

+
  private func traverseTree(writer: BitWriter, nodeIndex h: Int, childIndex child: Int) {
+    if tree[h].parent != -1 {
+      traverseTree(writer: writer, nodeIndex: tree[h].parent, childIndex: h)
+    }
+    if child != -1 {
+      if child == tree[h].left {
+        writer.writeBit(bit: true)
+      } else if child == tree[h].right {
+        writer.writeBit(bit: false)
+      }
+    }
+  }
+

When we call this method from compressData(), the nodeIndex parameter is the array index of the leaf node for the byte that we need to encode. This method recursively walks the tree from a leaf node up to the root and then back again.

+

As we are going back from the root to the leaf node, we write a 1 bit or a 0 bit for every node we encounter. If a child is the left node, we emit a 1; if it is the right node, we emit a 0.

+

In a picture:

+

How compression works

+

Even though the illustration of the tree shows a 0 or 1 for each edge between the nodes, the bit values 0 and 1 are not actually stored in the tree! The rule is that we write a 1 bit if we take the left branch and a 0 bit if we take the right branch, so just knowing the direction we are going in is enough to determine what bit value to write.

+

You use the compressData() method as follows:

+
let s1 = "so much words wow many compression"
+if let originalData = s1.dataUsingEncoding(NSUTF8StringEncoding) {
+  let huffman1 = Huffman()
+  let compressedData = huffman1.compressData(originalData)
+  print(compressedData.length)
+}
+

Decompression

+

Decompression is the compression in reverse. However, the compressed bits are useless without the frequency table. As mentioned, the frequencyTable() method returns an array of Freq objects. If we were saving the compressed data into a file or sending it across the network, we'd also save that [Freq] array along with it.

+

We first need some way to turn the [Freq] array back into a compression tree:

+
  fileprivate func restoreTree(fromTable frequencyTable: [Freq]) {
+    for freq in frequencyTable {
+      let i = Int(freq.byte)
+      tree[i].count = freq.count
+      tree[i].index = i
+    }
+    buildTree()
+  }
+

We convert the Freq objects into leaf nodes and then call buildTree() to do the rest.

+

Here is the code for decompressData(), which takes an NSData object with Huffman-encoded bits and a frequency table, and it returns the original data:

+
  func decompressData(data: NSData, frequencyTable: [Freq]) -> NSData {
+    restoreTree(fromTable: frequencyTable)
+
+    let reader = BitReader(data: data)
+    let outData = NSMutableData()
+    let byteCount = tree[root].count
+
+    var i = 0
+    while i < byteCount {
+      var b = findLeafNode(reader: reader, nodeIndex: root)
+      outData.append(&b, length: 1)
+      i += 1
+    }
+    return outData
+  }
+

This also uses a helper method to traverse the tree:

+
  private func findLeafNode(reader reader: BitReader, nodeIndex: Int) -> UInt8 {
+    var h = nodeIndex
+    while tree[h].right != -1 {
+      if reader.readBit() {
+        h = tree[h].left
+      } else {
+        h = tree[h].right
+      }
+    }
+    return UInt8(h)
+  }
+

findLeafNode() walks the tree from the root down to the leaf node given by nodeIndex. At each intermediate node, we read a new bit and then step to the left (bit is 1) or the right (bit is 0). When we get to the leaf node, we simply return its index, which is equal to the original byte value.

+

In a picture:

+

How decompression works

+

Here is how we use the decompression method:

+
  let frequencyTable = huffman1.frequencyTable()
+
+  let huffman2 = Huffman()
+  let decompressedData = huffman2.decompressData(compressedData, frequencyTable: frequencyTable)
+
+  let s2 = String(data: decompressedData, encoding: NSUTF8StringEncoding)!
+

First we get the frequency table from somewhere (in this case the Huffman object we used to encode the data) and then call decompressData(). The string that results should be equal to the one we compressed in the first place.

+

we can see how this works in more detail in the Playground.

+

See also

+

Huffman coding at Wikipedia

+

The code is loosely based on Al Stevens' C Programming column from Dr.Dobb's Magazine, February 1991 and October 1992.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Insertion Sort/index.html b/Insertion Sort/index.html new file mode 100644 index 000000000..e051ed035 --- /dev/null +++ b/Insertion Sort/index.html @@ -0,0 +1,170 @@ + + + Insertion Sort + + + +

Insertion Sort

+

Goal: Sort an array from low to high (or high to low).

+

You are given an array of numbers and need to put them in the right order. The insertion sort algorithm works as follows:

+ +

That's why this is called an "insertion" sort, because you take a number from the pile and insert it in the array in its proper sorted position.

+

An example

+

Let's say the numbers to sort are [ 8, 3, 5, 4, 6 ]. This is our unsorted pile.

+

Pick the first number, 8, and insert it into the new array. There is nothing in that array yet, so that's easy. The sorted array is now [ 8 ] and the pile is [ 3, 5, 4, 6 ].

+

Pick the next number from the pile, 3, and insert it into the sorted array. It should go before the 8, so the sorted array is now [ 3, 8 ] and the pile is reduced to [ 5, 4, 6 ].

+

Pick the next number from the pile, 5, and insert it into the sorted array. It goes in between the 3 and 8. The sorted array is [ 3, 5, 8 ] and the pile is [ 4, 6 ].

+

Repeat this process until the pile is empty.

+

In-place sort

+

The above explanation makes it seem like you need two arrays: one for the unsorted pile and one that contains the numbers in sorted order.

+

But you can perform the insertion sort in-place, without having to create a separate array. You just keep track of which part of the array is sorted already and which part is the unsorted pile.

+

Initially, the array is [ 8, 3, 5, 4, 6 ]. The | bar shows where the sorted portion ends and the pile begins:

+
[| 8, 3, 5, 4, 6 ]
+
+

This shows that the sorted portion is empty and the pile starts at 8.

+

After processing the first number, we have:

+
[ 8 | 3, 5, 4, 6 ]
+
+

The sorted portion is [ 8 ] and the pile is [ 3, 5, 4, 6 ]. The | bar has shifted one position to the right.

+

This is how the content of the array changes during the sort:

+
[| 8, 3, 5, 4, 6 ]
+[ 8 | 3, 5, 4, 6 ]
+[ 3, 8 | 5, 4, 6 ]
+[ 3, 5, 8 | 4, 6 ]
+[ 3, 4, 5, 8 | 6 ]
+[ 3, 4, 5, 6, 8 |]
+
+

In each step, the | bar moves up one position. As you can see, the beginning of the array up to the | is always sorted. The pile shrinks by one and the sorted portion grows by one, until the pile is empty and there are no more unsorted numbers left.

+

How to insert

+

At each step you pick the top-most number from the unsorted pile and insert it into the sorted portion of the array. You must put that number in the proper place so that the beginning of the array remains sorted. How does that work?

+

Let's say we've already done the first few elements and the array looks like this:

+
[ 3, 5, 8 | 4, 6 ]
+
+

The next number to sort is 4. We need to insert that into the sorted portion [ 3, 5, 8 ] somewhere.

+

Here's one way to do this: Look at the previous element, 8.

+
[ 3, 5, 8, 4 | 6 ]
+        ^
+
+

Is this greater than 4? Yes it is, so the 4 should come before the 8. We swap these two numbers to get:

+
[ 3, 5, 4, 8 | 6 ]
+        <-->
+       swapped
+
+

We're not done yet. The new previous element, 5, is also greater than 4. We also swap these two numbers:

+
[ 3, 4, 5, 8 | 6 ]
+     <-->
+    swapped
+
+

Again, look at the previous element. Is 3 greater than 4? No, it is not. That means we're done with number 4. The beginning of the array is sorted again.

+

This was a description of the inner loop of the insertion sort algorithm, which you'll see in the next section. It inserts the number from the top of the pile into the sorted portion by swapping numbers.

+

The code

+

Here is an implementation of insertion sort in Swift:

+
func insertionSort(_ array: [Int]) -> [Int] {
+    var a = array			 // 1
+    for x in 1..<a.count {		 // 2
+        var y = x
+        while y > 0 && a[y] < a[y - 1] { // 3
+            a.swapAt(y - 1, y)
+            y -= 1
+        }
+    }
+    return a
+}
+
+
+

Put this code in a playground and test it like so:

+
let list = [ 10, -1, 3, 9, 2, 27, 8, 5, 1, 3, 0, 26 ]
+insertionSort(list)
+

Here is how the code works.

+
    +
  1. +

    Make a copy of the array. This is necessary because we cannot modify the contents of the array parameter directly. Like Swift's own sort(), the insertionSort() function will return a sorted copy of the original array.

    +
    2. +
  2. +

    There are two loops inside this function. The outer loop looks at each of the elements in the array in turn; this is what picks the top-most number from the pile. The variable x is the index of where the sorted portion ends and the pile begins (the position of the | bar). Remember, at any given moment the beginning of the array -- from index 0 up to x -- is always sorted. The rest, from index x until the last element, is the unsorted pile.

    +
    3. +
  3. +

    The inner loop looks at the element at position x. This is the number at the top of the pile, and it may be smaller than any of the previous elements. The inner loop steps backwards through the sorted array; every time it finds a previous number that is larger, it swaps them. When the inner loop completes, the beginning of the array is sorted again, and the sorted portion has grown by one element.

    +
    4. +
+
+

Note: The outer loop starts at index 1, not 0. Moving the very first element from the pile to the sorted portion doesn't actually change anything, so we might as well skip it.

+
+

No more swaps

+

The above version of insertion sort works fine, but it can be made a tiny bit faster by removing the call to swap().

+

You've seen that we swap numbers to move the next element into its sorted position:

+
[ 3, 5, 8, 4 | 6 ]
+        <-->
+        swap
+        
+[ 3, 5, 4, 8 | 6 ]
+     <-->
+     swap
+
+

Instead of swapping with each of the previous elements, we can just shift all those elements one position to the right, and then copy the new number into the right position.

+
[ 3, 5, 8, 4 | 6 ]   remember 4
+           *
+
+[ 3, 5, 8, 8 | 6 ]   shift 8 to the right
+        --->
+        
+[ 3, 5, 5, 8 | 6 ]   shift 5 to the right
+     --->
+     
+[ 3, 4, 5, 8 | 6 ]   copy 4 into place
+     *
+
+

In code that looks like this:

+
func insertionSort(_ array: [Int]) -> [Int] {
+  var a = array
+  for x in 1..<a.count {
+    var y = x
+    let temp = a[y]
+    while y > 0 && temp < a[y - 1] {
+      a[y] = a[y - 1]                // 1
+      y -= 1
+    }
+    a[y] = temp                      // 2
+  }
+  return a
+}
+

The line at //1 is what shifts up the previous elements by one position. At the end of the inner loop, y is the destination index for the new number in the sorted portion, and the line at //2 copies this number into place.

+

Making it generic

+

It would be nice to sort other things than just numbers. We can make the datatype of the array generic and use a user-supplied function (or closure) to perform the less-than comparison. This only requires two changes to the code.

+

The function signature becomes:

+
func insertionSort<T>(_ array: [T], _ isOrderedBefore: (T, T) -> Bool) -> [T] {
+

The array has type [T] where T is the placeholder type for the generics. Now insertionSort() will accept any kind of array, whether it contains numbers, strings, or something else.

+

The new parameter isOrderedBefore: (T, T) -> Bool is a function that takes two T objects and returns true if the first object comes before the second, and false if the second object should come before the first. This is exactly what Swift's built-in sort() function does.

+

The only other change is in the inner loop, which now becomes:

+
      while y > 0 && isOrderedBefore(temp, a[y - 1]) {
+

Instead of writing temp < a[y - 1], we call the isOrderedBefore() function. It does the exact same thing, except we can now compare any kind of object, not just numbers.

+

To test this in a playground, do:

+
let numbers = [ 10, -1, 3, 9, 2, 27, 8, 5, 1, 3, 0, 26 ]
+insertionSort(numbers, <)
+insertionSort(numbers, >)
+

The < and > determine the sort order, low-to-high and high-to-low, respectively.

+

Of course, you can also sort other things such as strings,

+
let strings = [ "b", "a", "d", "c", "e" ]
+insertionSort(strings, <)
+

or even more complex objects:

+
let objects = [ obj1, obj2, obj3, ... ]
+insertionSort(objects) { $0.priority < $1.priority }
+

The closure tells insertionSort() to sort on the priority property of the objects.

+

Insertion sort is a stable sort. A sort is stable when elements that have identical sort keys remain in the same relative order after sorting. This is not important for simple values such as numbers or strings, but it is important when sorting more complex objects. In the example above, if two objects have the same priority, regardless of the values of their other properties, those two objects don't get swapped around.

+

Performance

+

Insertion sort is really fast if the array is already sorted. That sounds obvious, but this is not true for all search algorithms. In practice, a lot of data will already be largely -- if not entirely -- sorted and insertion sort works quite well in that case.

+

The worst-case and average case performance of insertion sort is O(n^2). That's because there are two nested loops in this function. Other sort algorithms, such as quicksort and merge sort, have O(n log n) performance, which is faster on large inputs.

+

Insertion sort is actually very fast for sorting small arrays. Some standard libraries have sort functions that switch from a quicksort to insertion sort when the partition size is 10 or less.

+

I did a quick test comparing our insertionSort() with Swift's built-in sort(). On arrays of about 100 items or so, the difference in speed is tiny. However, as your input becomes larger, O(n^2) quickly starts to perform a lot worse than O(n log n) and insertion sort just can't keep up.

+

See also

+

Insertion sort on Wikipedia

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Introsort/index.html b/Introsort/index.html new file mode 100644 index 000000000..87e51d6bb --- /dev/null +++ b/Introsort/index.html @@ -0,0 +1,88 @@ + + + Introsort + + + +

IntroSort

+

Goal: Sort an array from low to high (or high to low).

+

IntroSort is the algorithm used by swift to sort a collection. Introsort is an hybrid algorithm invented by David Musser in 1993 with the purpose of giving a generic sorting algorithm for the C++ standard library. The classic implementation of introsort expect a recursive Quicksort with fallback to Heapsort in case the recursion depth level reached a certain max. The maximum depends on the number of elements in the collection and it is usually 2 * log(n). The reason behind this “fallback” is that if Quicksort was not able to get the solution after 2 * log(n) recursions for a branch, probably it hit its worst case and it is degrading to complexity O( n^2 ). To optimise even further this algorithm, the swift implementation introduce an extra step in each recursion where the partition is sorted using InsertionSort if the count of the partition is less than 20.

+

The number 20 is an empiric number obtained observing the behaviour of InsertionSort with lists of this size.

+

Here's an implementation in pseudocode:

+
procedure sort(A : array):
+    let maxdepth = ⌊log(length(A))⌋ × 2
+    introSort(A, maxdepth)
+
+procedure introsort(A, maxdepth):
+    n ← length(A)
+    if n < 20:
+        insertionsort(A)
+    else if maxdepth = 0:
+        heapsort(A)
+    else:
+        p ← partition(A)  // the pivot is selected using median of 3
+        introsort(A[0:p], maxdepth - 1)
+        introsort(A[p+1:n], maxdepth - 1)
+
+

An example

+

Let's walk through the example. The array is initially:

+
[ 10, 0, 3, 9, 2, 14, 8, 27, 1, 5, 8, -1, 26 ]
+
+

For this example let's assume that maxDepth is 2 and that the size of the partition for the insertionSort to kick in is 5

+

The first iteration of introsort begins by attempting to use insertionSort. The collection has 13 elements, so it tries to do heapsort instead. The condition for heapsort to occur is if maxdepth == 0 evaluates true. Since maxdepth is currently 2 for the first iteration, introsort will default to quicksort.

+

The partition method picks the first element, the median and the last, it sorts them and uses the new median as pivot.

+
[ 10, 8, 26 ] -> [ 8, 10, 26 ]
+
+

Our array is now

+
[ 8, 0, 3, 9, 2, 14, 10, 27, 1, 5, 8, -1, 26 ]
+
+

10 is the pivot. After the choice of the pivot, the partition method swaps elements to get all the elements less than pivot on the left, and all the elements more or equal than pivot on the right.

+
[ 8, 0, 3, 9, 2, 1, 5, 8, -1, 10, 27, 14, 26 ]
+
+

Because of the swaps, the index of of pivot is now changed and returned. The next step of introsort is to call recursively itself for the two sub arrays:

+
less: [ 8, 0, 3, 9, 2, 1, 5, 8, -1, 10 ]
+greater: [ 27, 14, 26 ]
+
+

maxDepth: 1, branch: less

+
[ 8, 0, 3, 9, 2, 1, 5, 8, -1, 10 ]
+
+

The count of the array is still more than 5 so we don't meet yet the conditions for insertion sort to kick in. At this iteration maxDepth is decreased by one but it is still more than zero, so heapsort will not act.

+

Just like in the previous iteration quicksort wins and the partition method choses a pivot and sorts the elemets less than pivot on the left and the elements more or equeal than pivot on the right.

+
array: [ 8, 0, 3, 9, 2, 1, 5, 8, -1, 10 ]
+pivot candidates: [ 8, 1, 10] -> [ 1, 8, 10]
+pivot: 8
+before partition: [ 1, 0, 3, 9, 2, 8, 5, 8, -1, 10 ]
+after partition: [ 1, 0, 3, -1, 2, 5, 8, 8, 9, 10 ]
+
+less: [ 1, 0, 3, -1, 2, 5, 8 ]
+greater: [ 8, 9, 10 ]
+
+

maxDepth: 0, branch: less

+
[ 1, 0, 3, -1, 2, 5, 8 ]
+
+

Just like before, introsort is recursively executed for less and greater. This time lesshas a count more than 5 so it will not be sorted with insertion sort, but the maxDepth decreased again by 1 is now 0 and heapsort takes over sorting the array.

+
heapsort -> [ -1, 0, 1, 2, 3, 5, 8 ]
+
+

maxDepth: 0, branch: greater

+
[ 8, 9, 10 ]
+
+

following greater in this recursion, the count of elements is 3, which is less than 5, so this partition is sorted using insertionSort.

+
insertionSort -> [ 8, 9 , 10]
+
+

back to maxDepth = 1, branch: greater

+
[ 27, 14, 26 ]
+
+

At this point the original array has mutated to be

+
[ -1, 0, 1, 2, 3, 5, 8, 8, 9, 10, 27, 14, 26 ]
+
+

now the less partition is sorted and since the count of the greater partition is 3 it will be sorted with insertion sort [ 14, 26, 27 ]

+

The array is now successfully sorted

+
[ -1, 0, 1, 2, 3, 5, 8, 8, 9, 10, 14, 26, 27 ]
+
+

See also

+

Introsort on Wikipedia
+Introsort comparison with other sorting algorithms
+Introsort implementation from the Swift standard library

+

Written for Swift Algorithm Club by Giuseppe Lanza

+ + diff --git a/K-Means/index.html b/K-Means/index.html new file mode 100644 index 000000000..86a3f2ca7 --- /dev/null +++ b/K-Means/index.html @@ -0,0 +1,105 @@ + + + K-Means + + + +

k-Means Clustering

+

Goal: Partition data into two or more clusters.

+

The idea behind k-Means Clustering is to take a bunch of data and determine if there are any natural clusters (groups of related objects) within the data.

+

The k-Means algorithm is a so-called unsupervised learning algorithm. We don't know in advance what patterns exist in the data -- it has no formal classification to it -- but we would like to see if we can divide the data into groups somehow.

+

For example, you can use k-Means to find what are the 3 most prominent colors in an image by telling it to group pixels into 3 clusters based on their color value. Or you can use it to group related news articles together, without deciding beforehand what categories to use. The algorithm will automatically figure out what the best groups are.

+

The "k" in k-Means is a number. The algorithm assumes that there are k centers within the data that the various data elements are scattered around. The data that is closest to these so-called centroids become classified or grouped together.

+

k-Means doesn't tell you what the classifier is for each particular data group. After dividing news articles into groups, it doesn't say that group 1 is about science, group 2 is about celebrities, group 3 is about the upcoming election, etc. You only know that related news stories are now together, but not necessarily what that relationship signifies. k-Means only assists in trying to find what clusters potentially exist.

+

The algorithm

+

The k-Means algorithm is really quite simple at its core.

+

First, we choose k random data points to be the initial centroids. Then, we repeat the following two steps until we've found our clusters:

+
    +
  1. For each data point, find which centroid it is closest to. We assign each point to its nearest centroid.
    2. +
  2. Update the centroid to the mean (i.e. the average) of its nearest data points. We move the centroid so that it really sits in the center of the cluster.
    3. +
+

We need to repeat this multiple times because moving the centroid changes which data points belong to it. This goes back and forth for a bit until everything stabilizes. That's when we reach "convergence", i.e. when the centroids no longer move around.

+

A few of the parameters that are required for k-Means:

+ +

Let's look at an example.

+

Good clusters

+

This first example shows k-Means finding all three clusters. In all these examples the circles represent the data points and the stars represent the centroids.

+

In the first iteration, we choose three data points at random and put our centroids on top of them. Then in each subsequent iteration, we figure out which data points are closest to these centroids, and move the centroids to the average position of those data points. This repeats until we reach equilibrium and the centroids stop moving.

+

Good Clustering

+

The selection of initial centroids was fortuitous! We found the lower left cluster (indicated by red) and did pretty good on the center and upper left clusters.

+
+

Note: These examples are contrived to show the exact nature of k-Means and finding clusters. The clusters in these examples are very easily identified by human eyes: we see there is one in the lower left corner, one in the upper right corner, and maybe one in the middle. In practice, however, data may have many dimensions and may be impossible to visualize. In such cases, k-Means is much better at this job than human eyes!

+
+

Bad clustering

+

The next two examples highlight the unpredictability of k-Means and how it does not always find the best clustering.

+

Bad Clustering 1

+

As you can see in this example, the initial centroids were all a little too close to one another, and the blue one didn't quite get to a good place. By adjusting the convergence distance we should be able to improve the fit of our centroids to the data.

+

Bad Clustering 1

+

In this example, the blue cluster never really could separate from the red cluster and as such sort of got stuck down there.

+

Improving bad clustering

+

In these examples of "bad" clustering, the algorithm got stuck in a local optimum. It does find clusters but they're not the best way to divide up the data. To increase your chances of success, you can run the algorithm several times, each time with different points as the initial centroids. You choose the clustering that gives the best results.

+

To calculate how "good" the clustering is, you find the distance of each data point to its cluster, and add up all these distances. The lower this number, the better! That means each cluster is really in the center of a group of data points, and all clusters are roughly the same size and are spaced evenly apart.

+

The code

+

This is what the algorithm could look like in Swift. The points array contains the input data as Vector objects. The output is an array of Vector objects representing the clusters that were found.

+
func kMeans(numCenters: Int, convergeDistance: Double, points: [Vector]) -> [Vector] {
+ 
+  // Randomly take k objects from the input data to make the initial centroids.
+  var centers = reservoirSample(points, k: numCenters)
+
+  // This loop repeats until we've reached convergence, i.e. when the centroids
+  // have moved less than convergeDistance since the last iteration.
+  var centerMoveDist = 0.0
+  repeat {
+    // In each iteration of the loop, we move the centroids to a new position.
+    // The newCenters array contains those new positions.
+    let zeros = [Double](count: points[0].length, repeatedValue: 0)
+    var newCenters = [Vector](count: numCenters, repeatedValue: Vector(zeros))
+
+    // We keep track of how many data points belong to each centroid, so we
+    // can calculate the average later.
+    var counts = [Double](count: numCenters, repeatedValue: 0)
+
+    // For each data point, find the centroid that it is closest to. We also 
+    // add up the data points that belong to that centroid, in order to compute
+    // that average.
+    for p in points {
+      let c = indexOfNearestCenter(p, centers: centers)
+      newCenters[c] += p
+      counts[c] += 1
+    }
+    
+    // Take the average of all the data points that belong to each centroid.
+    // This moves the centroid to a new position.
+    for idx in 0..<numCenters {
+      newCenters[idx] /= counts[idx]
+    }
+
+    // Find out how far each centroid moved since the last iteration. If it's
+    // only a small distance, then we're done.
+    centerMoveDist = 0.0
+    for idx in 0..<numCenters {
+      centerMoveDist += centers[idx].distanceTo(newCenters[idx])
+    }
+    
+    centers = newCenters
+  } while centerMoveDist > convergeDistance
+
+  return centers
+}
+
+

Note: The code in KMeans.swift is slightly more advanced than the above listing. It also assigns labels to the clusters and has a few other tricks up its sleeve. Check it out!

+
+

Performance

+

k-Means is classified as an NP-Hard type of problem. That means it's almost impossible to find the optimal solution. The selection of the initial centroids has a big effect on how the resulting clusters may end up. Finding an exact solution is not likely -- even in 2 dimensional space.

+

As seen from the steps above the complexity really isn't that bad -- it is often considered to be on the order of O(kndi), where k is the number of centroids, n is the number of d-dimensional vectors, and i is the number of iterations for convergence.

+

The amount of data has a linear effect on the running time of k-Means, but tuning how far you want the centroids to converge can have a big impact how many iterations will be done. As a general rule, k should be relatively small compared to the number of vectors.

+

Often times as more data is added certain points may lie in the boundary between two centroids and as such those centroids would continue to bounce back and forth and the convergence distance would need to be tuned to prevent that.

+

See Also

+

K-Means Clustering on Wikipedia

+

Written by John Gill and Matthijs Hollemans

+ + diff --git a/Karatsuba Multiplication/index.html b/Karatsuba Multiplication/index.html new file mode 100644 index 000000000..39ed24636 --- /dev/null +++ b/Karatsuba Multiplication/index.html @@ -0,0 +1,105 @@ + + + Karatsuba Multiplication + + + +

Karatsuba Multiplication

+

Goal: To quickly multiply two numbers together

+

Long Multiplication

+

In grade school we learned how to multiply two numbers together via Long Multiplication. Let's try that first!

+

Example 1: Multiply 1234 by 5678 using Long Multiplication

+
    5678
+   *1234
+  ------
+   22712
+  17034-
+ 11356--
+ 5678---
+--------
+ 7006652
+
+

So what's the problem with Long Multiplication? Well remember the first part of our goal. To quickly multiply two numbers together. Long Multiplication is slow! (O(n^2))

+

You can see where the O(n^2) comes from in the implementation of Long Multiplication:

+
// Long Multiplication
+func multiply(_ num1: Int, by num2: Int, base: Int = 10) -> Int {
+  let num1Array = String(num1).characters.reversed().map{ Int(String($0))! }
+  let num2Array = String(num2).characters.reversed().map{ Int(String($0))! }
+  
+  var product = Array(repeating: 0, count: num1Array.count + num2Array.count)
+
+  for i in num1Array.indices {
+    var carry = 0
+    for j in num2Array.indices {
+      product[i + j] += carry + num1Array[i] * num2Array[j]
+      carry = product[i + j] / base
+      product[i + j] %= base
+    }
+    product[i + num2Array.count] += carry
+  }
+  
+  return Int(product.reversed().map{ String($0) }.reduce("", +))!
+}
+

The double for loop is the culprit! By comparing each of the digits (as is necessary!) we set ourselves up for an O(n^2) running time. So Long Multiplication might not be the best algorithm after all. Can we do better?

+

Karatsuba Multiplication

+

The Karatsuba Algorithm was discovered by Anatoly Karatsuba and published in 1962. Karatsuba discovered that you could compute the product of two large numbers using three smaller products and some addition and subtraction.

+

For two numbers x, y, where m <= n:

+
x = a*10^m + b
+y = c*10^m + d
+
+

Now, we can say:

+
x*y = (a*10^m + b) * (c*10^m + d)
+    = a*c*10^(2m) + (a*d + b*c)*10^(m) + b*d
+
+

This had been know since the 19th century. The problem is that the method requires 4 multiplications (a*c, a*d, b*c, b*d). Karatsuba's insight was that you only need three! (a*c, b*d, (a+b)*(c+d)). Now a perfectly valid question right now would be "How is that possible!?!" Here's the math:

+
    (a+b)*(c+d) - a*c - b*d  = (a*c + a*d + b*c + b*d) - a*c - b*d
+                             = (a*d + b*c)
+
+

Pretty cool, huh?

+

Here's the full implementation. Note that the recursive algorithm is most efficient at m = n/2.

+
// Karatsuba Multiplication
+func karatsuba(_ num1: Int, by num2: Int) -> Int {
+  let num1String = String(num1)
+  let num2String = String(num2)
+  
+  guard num1String.count > 1 && num2String.count > 1 else {
+    return multiply(num1, by: num2)
+  }
+  
+  let n = max(num1String.count, num2String.count)
+  let nBy2 = n / 2
+  
+  let a = num1 / 10^^nBy2
+  let b = num1 % 10^^nBy2
+  let c = num2 / 10^^nBy2
+  let d = num2 % 10^^nBy2
+  
+  let ac = karatsuba(a, by: c)
+  let bd = karatsuba(b, by: d)
+  let adPlusbc = karatsuba(a+b, by: c+d) - ac - bd
+  
+  let product = ac * 10^^(2 * nBy2) + adPlusbc * 10^^nBy2 + bd
+  
+  return product
+}
+

What about the running time of this algorithm? Is all this extra work worth it? We can use the Master Theorem to answer this question. This leads us to T(n) = 3*T(n/2) + c*n + d where c & d are some constants. It follows (because 3 > 2^1) that the running time is O(n^log2(3)) which is roughly O(n^1.56). Much better!

+

Example 2: Multiply 1234 by 5678 using Karatsuba Multiplication

+
m = 2
+x = 1234 = a*10^2 + b = 12*10^2 + 34
+y = 5678 = c*10^2 + d = 56*10^2 + 78
+
+a*c = 672
+b*d = 2652
+(a*d + b*c) = 2840
+
+x*y = 672*10^4 + 2840*10^2 + 2652
+    = 6720000 + 284000 + 2652
+    = 7006652	
+
+

Resources

+

Wikipedia

+

WolframMathWorld

+

Master Theorem

+

Written for Swift Algorithm Club by Richard Ash

+ + diff --git a/Knuth-Morris-Pratt/index.html b/Knuth-Morris-Pratt/index.html new file mode 100644 index 000000000..a72aebbf8 --- /dev/null +++ b/Knuth-Morris-Pratt/index.html @@ -0,0 +1,137 @@ + + + Knuth-Morris-Pratt + + + +

Knuth-Morris-Pratt String Search

+

Goal: Write a linear-time string matching algorithm in Swift that returns the indexes of all the occurrencies of a given pattern.

+

In other words, we want to implement an indexesOf(pattern: String) extension on String that returns an array [Int] of integers, representing all occurrences' indexes of the search pattern, or nil if the pattern could not be found inside the string.

+

For example:

+
let dna = "ACCCGGTTTTAAAGAACCACCATAAGATATAGACAGATATAGGACAGATATAGAGACAAAACCCCATACCCCAATATTTTTTTGGGGAGAAAAACACCACAGATAGATACACAGACTACACGAGATACGACATACAGCAGCATAACGACAACAGCAGATAGACGATCATAACAGCAATCAGACCGAGCGCAGCAGCTTTTAAGCACCAGCCCCACAAAAAACGACAATFATCATCATATACAGACGACGACACGACATATCACACGACAGCATA"
+dna.indexesOf(ptnr: "CATA")   // Output: [20, 64, 130, 140, 166, 234, 255, 270]
+
+let concert = "🎼🎹🎹🎸🎸🎻🎻🎷🎺🎤👏👏👏"
+concert.indexesOf(ptnr: "🎻🎷")   // Output: [6]
+

The Knuth-Morris-Pratt algorithm is considered one of the best algorithms for solving the pattern matching problem. Although in practice Boyer-Moore is usually preferred, the algorithm that we will introduce is simpler, and has the same (linear) running time.

+

The idea behind the algorithm is not too different from the naive string search procedure. As it, Knuth-Morris-Pratt aligns the text with the pattern and goes with character comparisons from left to right. But, instead of making a shift of one character when a mismatch occurs, it uses a more intelligent way to move the pattern along the text. In fact, the algorithm features a pattern pre-processing stage where it acquires all the informations that will make the algorithm skip redundant comparisons, resulting in larger shifts.

+

The pre-processing stage produces an array (called suffixPrefix in the code) of integers in which every element suffixPrefix[i] records the length of the longest proper suffix of P[0...i] (where P is the pattern) that matches a prefix of P. In other words, suffixPrefix[i] is the longest proper substring of P that ends at position i and that is a prefix of P. Just a quick example. Consider P = "abadfryaabsabadffg", then suffixPrefix[4] = 0, suffixPrefix[9] = 2, suffixPrefix[14] = 4.
+There are different ways to obtain the values of SuffixPrefix array. We will use the method based on the Z-Algorithm. This function takes in input the pattern and produces an array of integers. Each element represents the length of the longest substring starting at position i of P and that matches a prefix of P. You can notice that the two arrays are similar, they record the same informations but on the different places. We only have to find a method to map Z[i] to suffixPrefix[j]. It is not that difficult and this is the code that will do for us:

+
for patternIndex in (1 ..< patternLength).reversed() {
+    textIndex = patternIndex + zeta![patternIndex] - 1
+    suffixPrefix[textIndex] = zeta![patternIndex]
+}
+

We are simply computing the index of the end of the substring starting at position i (as we know matches a prefix of P). The element of suffixPrefix at that index then it will be set with the length of the substring.

+

Once the shift-array suffixPrefix is ready we can begin with pattern search stage. The algorithm first attempts to compare the characters of the text with those of the pattern. If it succeeds, it goes on until a mismatch occurs. When it happens, it checks if an occurrence of the pattern is present (and reports it). Otherwise, if no comparisons are made then the text cursor is moved forward, else the pattern is shifted to the right. The shift's amount is based on the suffixPrefix array, and it guarantees that the prefix P[0...suffixPrefix[i]] will match its opposing substring in the text. In this way, shifts of more than one character are often made and lot of comparisons can be avoided, saving a lot of time.

+

Here is the code of the Knuth-Morris-Pratt algorithm:

+
extension String {
+
+    func indexesOf(ptnr: String) -> [Int]? {
+
+        let text = Array(self.characters)
+        let pattern = Array(ptnr.characters)
+
+        let textLength: Int = text.count
+        let patternLength: Int = pattern.count
+
+        guard patternLength > 0 else {
+            return nil
+        }
+
+        var suffixPrefix: [Int] = [Int](repeating: 0, count: patternLength)
+        var textIndex: Int = 0
+        var patternIndex: Int = 0
+        var indexes: [Int] = [Int]()
+
+        /* Pre-processing stage: computing the table for the shifts (through Z-Algorithm) */
+        let zeta = ZetaAlgorithm(ptnr: ptnr)
+
+        for patternIndex in (1 ..< patternLength).reversed() {
+            textIndex = patternIndex + zeta![patternIndex] - 1
+            suffixPrefix[textIndex] = zeta![patternIndex]
+        }
+
+        /* Search stage: scanning the text for pattern matching */
+        textIndex = 0
+        patternIndex = 0
+
+        while textIndex + (patternLength - patternIndex - 1) < textLength {
+
+            while patternIndex < patternLength && text[textIndex] == pattern[patternIndex] {
+                textIndex = textIndex + 1
+                patternIndex = patternIndex + 1
+            }
+
+            if patternIndex == patternLength {
+                indexes.append(textIndex - patternIndex)
+            }
+
+            if patternIndex == 0 {
+                textIndex = textIndex + 1
+            } else {
+                patternIndex = suffixPrefix[patternIndex - 1]
+            }
+        }
+
+        guard !indexes.isEmpty else {
+            return nil
+        }
+        return indexes
+    }
+}
+

Let's make an example reasoning with the code above. Let's consider the string P = ACTGACTA", the consequentially obtained suffixPrefix array equal to [0, 0, 0, 0, 0, 0, 3, 1], and the text T = "GCACTGACTGACTGACTAG". The algorithm begins with the text and the pattern aligned like below. We have to compare T[0] with P[0].

+
                          1       
+                0123456789012345678
+text:           GCACTGACTGACTGACTAG
+textIndex:      ^
+pattern:        ACTGACTA
+patternIndex:   ^
+                x
+suffixPrefix:   00000031
+
+

We have a mismatch and we move on comparing T[1] and P[0]. We have to check if a pattern occurrence is present but there is not. So, we have to shift the pattern right and by doing so we have to check suffixPrefix[1 - 1]. Its value is 0 and we restart by comparing T[1] with P[0]. Again a mismath occurs, so we go on with T[2] and P[0].

+
                          1      
+                0123456789012345678
+text:           GCACTGACTGACTGACTAG
+textIndex:        ^
+pattern:          ACTGACTA
+patternIndex:     ^
+suffixPrefix:     00000031
+
+

This time we have a match. And it continues until position 8. Unfortunately the length of the match is not equal to the pattern length, we cannot report an occurrence. But we are still lucky because we can use the values computed in the suffixPrefix array now. In fact, the length of the match is 7, and if we look at the element suffixPrefix[7 - 1] we discover that is 3. This information tell us that that the prefix of P matches the suffix of the susbtring T[0...8]. So the suffixPrefix array guarantees us that the two substring match and that we do not have to compare their characters, so we can shift right the pattern for more than one character!
+The comparisons restart from T[9] and P[3].

+
                          1       
+                0123456789012345678
+text:           GCACTGACTGACTGACTAG
+textIndex:               ^
+pattern:              ACTGACTA
+patternIndex:            ^
+suffixPrefix:         00000031
+
+

They match so we continue the compares until position 13 where a misatch occurs beetwen charcter G and A. Just like before, we are lucky and we can use the suffixPrefix array to shift right the pattern.

+
                          1       
+                0123456789012345678
+text:           GCACTGACTGACTGACTAG
+textIndex:                   ^
+pattern:                  ACTGACTA
+patternIndex:                ^
+suffixPrefix:             00000031
+
+

Again, we have to compare. But this time the comparisons finally take us to an occurrence, at position 17 - 7 = 10.

+
                          1       
+                0123456789012345678
+text:           GCACTGACTGACTGACTAG
+textIndex:                       ^
+pattern:                  ACTGACTA
+patternIndex:                    ^
+suffixPrefix:             00000031
+
+

The algorithm than tries to compare T[18] with P[1] (because we used the element suffixPrefix[8 - 1] = 1) but it fails and at the next iteration it ends its work.

+

The pre-processing stage involves only the pattern. The running time of the Z-Algorithm is linear and takes O(n), where n is the length of the pattern P. After that, the search stage does not "overshoot" the length of the text T (call it m). It can be be proved that number of comparisons of the search stage is bounded by 2 * m. The final running time of the Knuth-Morris-Pratt algorithm is O(n + m).

+
+

Note: To execute the code in the KnuthMorrisPratt.swift you have to copy the ZAlgorithm.swift file contained in the Z-Algorithm folder. The KnuthMorrisPratt.playground already includes the definition of the Zeta function.

+
+

Credits: This code is based on the handbook "Algorithm on String, Trees and Sequences: Computer Science and Computational Biology" by Dan Gusfield, Cambridge University Press, 1997.

+

Written for Swift Algorithm Club by Matteo Dunnhofer

+ + diff --git a/Kth Largest Element/index.html b/Kth Largest Element/index.html new file mode 100644 index 000000000..f792d4897 --- /dev/null +++ b/Kth Largest Element/index.html @@ -0,0 +1,114 @@ + + + Kth Largest Element + + + +

k-th Largest Element Problem

+

You're given an integer array a. Write an algorithm that finds the k-th largest element in the array.

+

For example, the 1-st largest element is the maximum value that occurs in the array. If the array has n elements, the n-th largest element is the minimum. The median is the n/2-th largest element.

+

The naive solution

+

The following solution is semi-naive. Its time complexity is O(n log n) since it first sorts the array, and therefore also uses additional O(n) space.

+
func kthLargest(a: [Int], k: Int) -> Int? {
+  let len = a.count
+  if k > 0 && k <= len {
+    let sorted = a.sorted()
+    return sorted[len - k]
+  } else {
+    return nil
+  }
+}
+

The kthLargest() function takes two parameters: the array a consisting of integers, and k. It returns the k-th largest element.

+

Let's take a look at an example and run through the algorithm to see how it works. Given k = 4 and the array:

+
[ 7, 92, 23, 9, -1, 0, 11, 6 ]
+

Initially there's no direct way to find the k-th largest element, but after sorting the array it's rather straightforward. Here's the sorted array:

+
[ -1, 0, 6, 7, 9, 11, 23, 92 ]
+

Now, all we must do is take the value at index a.count - k:

+
a[a.count - k] = a[8 - 4] = a[4] = 9
+

Of course, if you were looking for the k-th smallest element, you'd use a[k-1].

+

A faster solution

+

There is a clever algorithm that combines the ideas of binary search and quicksort to arrive at an O(n) solution.

+

Recall that binary search splits the array in half over and over again, to quickly narrow in on the value you're searching for. That's what we'll do here too.

+

Quicksort also splits up arrays. It uses partitioning to move all smaller values to the left of the pivot and all greater values to the right. After partitioning around a certain pivot, that pivot value will already be in its final, sorted position. We can use that to our advantage here.

+

Here's how it works: We choose a random pivot, partition the array around that pivot, then act like a binary search and only continue in the left or right partition. This repeats until we've found a pivot that happens to end up in the k-th position.

+

Let's look at the original example again. We're looking for the 4-th largest element in this array:

+
[ 7, 92, 23, 9, -1, 0, 11, 6 ]
+
+

The algorithm is a bit easier to follow if we look for the k-th smallest item instead, so let's take k = 4 and look for the 4-th smallest element.

+

Note that we don't have to sort the array first. We pick one of the elements at random to be the pivot, let's say 11, and partition the array around that. We might end up with something like this:

+
[ 7, 9, -1, 0, 6, 11, 92, 23 ]
+ <------ smaller    larger -->
+
+

As you can see, all values smaller than 11 are on the left; all values larger are on the right. The pivot value 11 is now in its final place. The index of the pivot is 5, so the 4-th smallest element must be in the left partition somewhere. We can ignore the rest of the array from now on:

+
[ 7, 9, -1, 0, 6, x, x, x ]
+
+

Again let's pick a random pivot, let's say 6, and partition the array around it. We might end up with something like this:

+
[ -1, 0, 6, 9, 7, x, x, x ]
+
+

Pivot 6 ended up at index 2, so obviously the 4-th smallest item must be in the right partition. We can ignore the left partition:

+
[ x, x, x, 9, 7, x, x, x ]
+
+

Again we pick a pivot value at random, let's say 9, and partition the array:

+
[ x, x, x, 7, 9, x, x, x ]
+
+

The index of pivot 9 is 4, and that's exactly the k we're looking for. We're done! Notice how this only took a few steps and we did not have to sort the array first.

+

The following function implements these ideas:

+
public func randomizedSelect<T: Comparable>(_ array: [T], order k: Int) -> T {
+  var a = array
+
+  func randomPivot<T: Comparable>(_ a: inout [T], _ low: Int, _ high: Int) -> T {
+    let pivotIndex = random(min: low, max: high)
+    a.swapAt(pivotIndex, high)
+    return a[high]
+  }
+
+  func randomizedPartition<T: Comparable>(_ a: inout [T], _ low: Int, _ high: Int) -> Int {
+    let pivot = randomPivot(&a, low, high)
+    var i = low
+    for j in low..<high {
+      if a[j] <= pivot {
+        a.swapAt(i, j)
+        i += 1
+      }
+    }
+    a.swapAt(i, high)
+    return i
+  }
+
+  func randomizedSelect<T: Comparable>(_ a: inout [T], _ low: Int, _ high: Int, _ k: Int) -> T {
+    if low < high {
+      let p = randomizedPartition(&a, low, high)
+      if k == p {
+        return a[p]
+      } else if k < p {
+        return randomizedSelect(&a, low, p - 1, k)
+      } else {
+        return randomizedSelect(&a, p + 1, high, k)
+      }
+    } else {
+      return a[low]
+    }
+  }
+
+  precondition(a.count > 0)
+  return randomizedSelect(&a, 0, a.count - 1, k)
+}
+

To keep things readable, the functionality is split into three inner functions:

+ +

Pretty cool, huh? Normally quicksort is an O(n log n) algorithm, but because we only partition smaller and smaller slices of the array, the running time of randomizedSelect() works out to O(n).

+
+

Note: This function calculates the k-th smallest item in the array, where k starts at 0. If you want the k-th largest item, call it with a.count - k.

+
+

Written by Daniel Speiser. Additions by Matthijs Hollemans.

+ + diff --git a/LRU Cache/index.html b/LRU Cache/index.html new file mode 100644 index 000000000..11559ae9e --- /dev/null +++ b/LRU Cache/index.html @@ -0,0 +1,40 @@ + + + LRU Cache + + + +

LRU Cache

+

Caches are used to hold objects in memory. A caches size is finite; If the system doesn't have enough memory, the cache must be purged or the program will crash. Least Recently Used (LRU) is a popular algorithm in cache design.

+

In this implementation of the LRU Cache, a size is declared during instantiation, and any insertions that go beyond the size will purge the least recently used element of the cache. A priority queue is used to enforce this behavior.

+

The priority queue

+

The key to the LRU cache is the priority queue. For simplicity, you'll model the queue using a linked list. All interactions with the LRU cache should respect this queue; Calling get and set should update the priority queue to reflect the most recently accessed element.

+

Interesting tidbits

+

Adding values

+

Each time we access an element, either set or get we need to insert the element in the head of priority list. We use a helper method to handle this procedure:

+
private func insert(_ key: KeyType, val: Any) {
+	cache[key] = val
+	priority.insert(key, atIndex: 0)
+	guard let first = priority.first else {
+		return
+	}
+	key2node[key] = first
+}
+

Purging the cache

+

When the cache is full, a purge must take place starting with the least recently used element. In this case, we need to remove the lowest priority node. The operation is like this:

+
private func remove(_ key: KeyType) {
+	cache.removeValue(key)
+	guard let node = key2node[key] else {
+		return
+	}
+	priority.remove(node)
+	key2node.removeValue(key)
+}
+

Optimizing Performance

+

Removing elements from the priority queue is a frequent operation for the LRU cache. Since priority queue is modelled using a linked list, this is an expensive operation that costs O(n) time. This is the bottleneck for both the set and get methods.

+

To help alleviate this problem, a hash table is used to store the references of each node:

+
private var key2node: [KeyType: LinkedList<KeyType>.LinkedListNode<KeyType>] = [:]
+
+

Written for the Swift Algorithm Club by Kai Chen, with additions by Kelvin Lau

+ + diff --git a/Linear Regression/index.html b/Linear Regression/index.html new file mode 100644 index 000000000..9ad4b8e4d --- /dev/null +++ b/Linear Regression/index.html @@ -0,0 +1,167 @@ + + + Linear Regression + + + +

Linear Regression

+

Linear regression is a technique for creating a model of the relationship between two (or more) variable quantities.

+

For example, let's say we are planning to sell a car. We are not sure how much money to ask for. So we look at recent advertisments for the asking prices of other cars. There are a lot of variables we could look at - for example: make, model, engine size. To simplify our task, we collect data on just the age of the car and the price:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Age (in years)Price (in £)
10500
8400
37,000
38,500
211,000
110,500
+

Our car is 4 years old. How can we set a price for our car based on the data in this table?

+

Let's start by looking at the data plotted out:

+

graph1

+

We could imagine a straight line drawn through the points on this graph. It's not (in this case) going to go exactly through every point, but we could place the line so that it goes as close to all the points as possible.

+

To say this in another way, we want to make the distance from the line to each point as small as possible. This is most often done by minimizing the square of the distance from the line to each point.

+

We can describe the straight line in terms of two variables:

+
    +
  1. The point at which it crosses the y-axis i.e. the predicted price of a brand new car. This is the intercept.
    2. +
  2. The slope of the line - i.e. for every year of age, how much does the price change.
    3. +
+

This is the equation for our line:

+

carPrice = slope * carAge + intercept

+

How can we find the best values for the intercept and the slope? Let's look at two different ways to do this.

+

An iterative approach

+

One approach is to start with some arbitrary values for the intercept and the slope. We work out what small changes we make to these values to move our line closer to the data points. Then we repeat this multiple times. Eventually our line will approach the optimum position.

+

First let's set up our data structures. We will use two Swift arrays for the car age and the car price:

+
let carAge: [Double] = [10, 8, 3, 3, 2, 1]
+let carPrice: [Double] = [500, 400, 7000, 8500, 11000, 10500]
+

This is how we can represent our straight line:

+
var intercept = 0.0
+var slope = 0.0
+func predictedCarPrice(_ carAge: Double) -> Double {
+    return intercept + slope * carAge
+}
+
+

Now for the code which will perform the iterations:

+
let numberOfCarAdvertsWeSaw = carPrice.count
+let numberOfIterations = 100
+let alpha = 0.0001
+
+for _ in 1...numberOfIterations {
+    for i in 0..<numberOfCarAdvertsWeSaw {
+        let difference = carPrice[i] - predictedCarPrice(carAge[i])
+        intercept += alpha * difference
+        slope += alpha * difference * carAge[i]
+    }
+}
+

alpha is a factor that determines how much closer we move to the correct solution with each iteration. If this factor is too large then our program will not converge on the correct solution.

+

The program loops through each data point (each car age and car price). For each data point it adjusts the intercept and the slope to bring them closer to the correct values. The equations used in the code to adjust the intercept and the slope are based on moving in the direction of the maximal reduction of these variables. This is a gradient descent.

+

We want to minimize the square of the distance between the line and the points. We define a function J which represents this distance - for simplicity we consider only one point here. This function J is proportional to ((slope * carAge + intercept) - carPrice)) ^ 2.

+

In order to move in the direction of maximal reduction, we take the partial derivative of this function with respect to the slope, and similarly for the intercept. We multiply these derivatives by our factor alpha and then use them to adjust the values of slope and intercept on each iteration.

+

Looking at the code, it intuitively makes sense - the larger the difference between the current predicted car Price and the actual car price, and the larger the value of alpha, the greater the adjustments to the intercept and the slope.

+

It can take a lot of iterations to approach the ideal values. Let's look at how the intercept and slope change as we increase the number of iterations:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IterationsInterceptSlopePredicted value of a 4 year old car
0000
20004112-1133659
60008564-7645507
1000010517-10496318
1400011374-11756673
1800011750-12306829
+

Here is the same data shown as a graph. Each of the blue lines on the graph represents a row in the table above.

+

graph2

+

After 18,000 iterations it looks as if the line is getting closer to what we would expect (just by looking) to be the correct line of best fit. Also, each additional 2,000 iterations has less and less effect on the final result - the values of the intercept and the slope are converging on the correct values.

+

A closed form solution

+

There is another way we can calculate the line of best fit, without having to do multiple iterations. We can solve the equations describing the least squares minimisation and just work out the intercept and slope directly.

+

First we need some helper functions. This one calculates the average (the mean) of an array of Doubles:

+
func average(_ input: [Double]) -> Double {
+    return input.reduce(0, +) / Double(input.count)
+}
+

We are using the reduce Swift function to sum up all the elements of the array, and then divide that by the number of elements. This gives us the mean value.

+

We also need to be able to multiply each element in an array by the corresponding element in another array, to create a new array. Here is a function which will do this:

+
func multiply(_ a: [Double], _ b: [Double]) -> [Double] {
+    return zip(a,b).map(*)
+}
+

We are using the map function to multiply each element.

+

Finally, the function which fits the line to the data:

+
func linearRegression(_ xs: [Double], _ ys: [Double]) -> (Double) -> Double {
+    let sum1 = average(multiply(ys, xs)) - average(xs) * average(ys)
+    let sum2 = average(multiply(xs, xs)) - pow(average(xs), 2)
+    let slope = sum1 / sum2
+    let intercept = average(ys) - slope * average(xs)
+    return { x in intercept + slope * x }
+}
+

This function takes as arguments two arrays of Doubles, and returns a function which is the line of best fit. The formulas to calculate the slope and the intercept can be derived from our definition of the function J. Let's see how the output from this line fits our data:

+

graph3

+

Using this line, we would predict a price for our 4 year old car of £6952.

+

Summary

+

We've seen two different ways to implement a simple linear regression in Swift. An obvious question is: why bother with the iterative approach at all?

+

Well, the line we've found doesn't fit the data perfectly. For one thing, the graph includes some negative values at high car ages! Possibly we would have to pay someone to tow away a very old car... but really these negative values just show that we have not modelled the real life situation very accurately. The relationship between the car age and the car price is not linear but instead is some other function. We also know that a car's price is not just related to its age but also other factors such as the make, model and engine size of the car. We would need to use additional variables to describe these other factors.

+

It turns out that in some of these more complicated models, the iterative approach is the only viable or efficient approach. This can also occur when the arrays of data are very large and may be sparsely populated with data values.

+

Written for Swift Algorithm Club by James Harrop

+ + diff --git a/Linear Search/index.html b/Linear Search/index.html new file mode 100644 index 000000000..8dc4a624e --- /dev/null +++ b/Linear Search/index.html @@ -0,0 +1,32 @@ + + + Linear Search + + + +

Linear Search

+

Goal: Find a particular value in an array.

+

We have an array of generic objects. With linear search, we iterate over all the objects in the array and compare each one to the object we're looking for. If the two objects are equal, we stop and return the current array index. If not, we continue to look for the next object as long as we have objects in the array.

+

An example

+

Let's say we have an array of numbers [5, 2, 4, 7] and we want to check if the array contains the number 2.

+

We start by comparing the first number in the array, 5, to the number we're looking for, 2. They are obviously not the same, and so we continue to the next array element.

+

We compare the number 2 from the array to our number 2 and notice they are equal. Now we can stop our iteration and return 1, which is the index of the number 2 in the array.

+

The code

+

Here is a simple implementation of linear search in Swift:

+
func linearSearch<T: Equatable>(_ array: [T], _ object: T) -> Int? {
+  for (index, obj) in array.enumerated() where obj == object {
+    return index
+  }
+  return nil
+}
+

Put this code in a playground and test it like so:

+
let array = [5, 2, 4, 7]
+linearSearch(array, 2) 	// This will return 1
+

Performance

+

Linear search runs at O(n). It compares the object we are looking for with each object in the array and so the time it takes is proportional to the array length. In the worst case, we need to look at all the elements in the array.

+

The best-case performance is O(1) but this case is rare because the object we're looking for has to be positioned at the start of the array to be immediately found. You might get lucky, but most of the time you won't. On average, linear search needs to look at half the objects in the array.

+

See also

+

Linear search on Wikipedia

+

Written by Patrick Balestra

+ + diff --git a/Linked List/index.html b/Linked List/index.html new file mode 100644 index 000000000..e17b79e30 --- /dev/null +++ b/Linked List/index.html @@ -0,0 +1,467 @@ + + + Linked List + + + +

Linked List

+
+

This topic has been tutorialized here

+
+

A linked list is a sequence of data items, just like an array. But where an array allocates a big block of memory to store the objects, the elements in a linked list are totally separate objects in memory and are connected through links:

+
+--------+    +--------+    +--------+    +--------+
+|        |    |        |    |        |    |        |
+| node 0 |--->| node 1 |--->| node 2 |--->| node 3 |
+|        |    |        |    |        |    |        |
++--------+    +--------+    +--------+    +--------+
+
+

The elements of a linked list are referred to as nodes. The above picture shows a singly linked list, where each node only has a reference -- or a "pointer" -- to the next node. In a doubly linked list, shown below, nodes also have pointers to the previous node:

+
+--------+    +--------+    +--------+    +--------+
+|        |--->|        |--->|        |--->|        |
+| node 0 |    | node 1 |    | node 2 |    | node 3 |
+|        |<---|        |<---|        |<---|        |
++--------+    +--------+    +--------+    +--------+
+
+

You need to keep track of where the list begins. That's usually done with a pointer called the head:

+
         +--------+    +--------+    +--------+    +--------+
+head --->|        |--->|        |--->|        |--->|        |---> nil
+         | node 0 |    | node 1 |    | node 2 |    | node 3 |
+ nil <---|        |<---|        |<---|        |<---|        |<--- tail
+         +--------+    +--------+    +--------+    +--------+
+
+

It's also useful to have a reference to the end of the list, known as the tail. Note that the "next" pointer of the last node is nil, just like the "previous" pointer of the very first node.

+

Performance of linked lists

+

Most operations on a linked list have O(n) time, so linked lists are generally slower than arrays. However, they are also much more flexible -- rather than having to copy large chunks of memory around as with an array, many operations on a linked list just require you to change a few pointers.

+

The reason for the O(n) time is that you can't simply write list[2] to access node 2 from the list. If you don't have a reference to that node already, you have to start at the head and work your way down to that node by following the next pointers (or start at the tail and work your way back using the previous pointers).

+

But once you have a reference to a node, operations like insertion and deletion are really quick. It's just that finding the node is slow.

+

This means that when you're dealing with a linked list, you should insert new items at the front whenever possible. That is an O(1) operation. Likewise for inserting at the back if you're keeping track of the tail pointer.

+

Singly vs doubly linked lists

+

A singly linked list uses a little less memory than a doubly linked list because it doesn't need to store all those previous pointers.

+

But if you have a node and you need to find its previous node, you're screwed. You have to start at the head of the list and iterate through the entire list until you get to the right node.

+

For many tasks, a doubly linked list makes things easier.

+

Why use a linked list?

+

A typical example of where to use a linked list is when you need a queue. With an array, removing elements from the front of the queue is slow because it needs to shift down all the other elements in memory. But with a linked list it's just a matter of changing head to point to the second element. Much faster.

+

But to be honest, you hardly ever need to write your own linked list these days. Still, it's useful to understand how they work; the principle of linking objects together is also used with trees and graphs.

+

The code

+

We'll start by defining a type to describe the nodes:

+
public class LinkedListNode<T> {
+  var value: T
+  var next: LinkedListNode?
+  weak var previous: LinkedListNode?
+
+  public init(value: T) {
+    self.value = value
+  }
+}
+

This is a generic type, so T can be any kind of data that you'd like to store in the node. We'll be using strings in the examples that follow.

+

Ours is a doubly-linked list and each node has a next and previous pointer. These can be nil if there are no next or previous nodes, so these variables must be optionals. (In what follows, I'll point out which functions would need to change if this was just a singly- instead of a doubly-linked list.)

+
+

Note: To avoid ownership cycles, we declare the previous pointer to be weak. If you have a node A that is followed by node B in the list, then A points to B but also B points to A. In certain circumstances, this ownership cycle can cause nodes to be kept alive even after you deleted them. We don't want that, so we make one of the pointers weak to break the cycle.

+
+

Let's start building LinkedList. Here's the first bit:

+
public class LinkedList<T> {
+  public typealias Node = LinkedListNode<T>
+
+  private var head: Node?
+
+  public var isEmpty: Bool {
+    return head == nil
+  }
+
+  public var first: Node? {
+    return head
+  }
+}
+

Ideally, we would want a class name to be as descriptive as possible, yet, we don't want to type a long name every time we want to use the class, therefore, we're using a typealias so inside LinkedList we can write the shorter Node instead of LinkedListNode<T>.

+

This linked list only has a head pointer, not a tail. Adding a tail pointer is left as an exercise for the reader. (I'll point out which functions would be different if we also had a tail pointer.)

+

The list is empty if head is nil. Because head is a private variable, I've added the property first to return a reference to the first node in the list.

+

You can try it out in a playground like this:

+
let list = LinkedList<String>()
+list.isEmpty   // true
+list.first     // nil
+

Let's also add a property that gives you the last node in the list. This is where it starts to become interesting:

+
  public var last: Node? {
+    guard var node = head else {
+      return nil
+    }
+  
+    while let next = node.next {
+      node = next
+    }
+    return node
+  }
+

If you're new to Swift, you've probably seen if let but maybe not if var. It does the same thing -- it unwraps the head optional and puts the result in a new local variable named node. The difference is that node is not a constant but an actual variable, so we can change it inside the loop.

+

The loop also does some Swift magic. The while let next = node.next bit keeps looping until node.next is nil. You could have written this as follows:

+
      var node: Node? = head
+      while node != nil && node!.next != nil {
+        node = node!.next
+      }
+

But that doesn't feel very Swifty to me. We might as well make use of Swift's built-in support for unwrapping optionals. You'll see a bunch of these kinds of loops in the code that follows.

+
+

Note: If we kept a tail pointer, last would simply do return tail. But we don't, so it has to step through the entire list from beginning to the end. It's an expensive operation, especially if the list is long.

+
+

Of course, last only returns nil because we don't have any nodes in the list. Let's add a method that adds a new node to the end of the list:

+
  public func append(value: T) {
+    let newNode = Node(value: value)
+    if let lastNode = last {
+      newNode.previous = lastNode
+      lastNode.next = newNode
+    } else {
+      head = newNode
+    }
+  }
+

The append() method first creates a new Node object and then asks for the last node using the last property we've just added. If there is no such node, the list is still empty and we make head point to this new Node. But if we did find a valid node object, we connect the next and previous pointers to link this new node into the chain. A lot of linked list code involves this kind of next and previous pointer manipulation.

+

Let's test it in the playground:

+
list.append("Hello")
+list.isEmpty         // false
+list.first!.value    // "Hello"
+list.last!.value     // "Hello"
+

The list looks like this:

+
         +---------+
+head --->|         |---> nil
+         | "Hello" |
+ nil <---|         |
+         +---------+
+
+

Now add a second node:

+
list.append("World")
+list.first!.value    // "Hello"
+list.last!.value     // "World"
+

And the list looks like:

+
         +---------+    +---------+
+head --->|         |--->|         |---> nil
+         | "Hello" |    | "World" |
+ nil <---|         |<---|         |
+         +---------+    +---------+
+
+

You can verify this for yourself by looking at the next and previous pointers:

+
list.first!.previous          // nil
+list.first!.next!.value       // "World"
+list.last!.previous!.value    // "Hello"
+list.last!.next               // nil
+

Let's add a method to count how many nodes are in the list. This will look very similar to what we did already:

+
  public var count: Int {
+    guard var node = head else {
+      return 0
+    }
+  
+    var count = 1
+    while let next = node.next {
+      node = next
+      count += 1
+    }
+    return count
+  }
+

It loops through the list in the same manner but this time increments a counter as well.

+
+

Note: One way to speed up count from O(n) to O(1) is to keep track of a variable that counts how many nodes are in the list. Whenever you add or remove a node, you also update this variable.

+
+

What if we wanted to find the node at a specific index in the list? With an array we can just write array[index] and it's an O(1) operation. It's a bit more involved with linked lists, but again the code follows a similar pattern:

+
  public func node(atIndex index: Int) -> Node {
+    if index == 0 {
+      return head!
+    } else {
+      var node = head!.next
+      for _ in 1..<index {
+        node = node?.next
+        if node == nil { //(*1)
+          break
+        }
+      }
+      return node!
+    }
+  }
+

First we check whether the given index is 0 or not. Because if it is 0, it returns the head as it is.
+However, when the given index is greater than 0, it starts at head then keeps following the node.next pointers to step through the list.
+The difference from count method at this time is that there are two termination conditions.
+One is when the for-loop statement reaches index, and we were able to acquire the node of the given index.
+The second is when node.next in for-loop statement returns nil and cause break. (*1)
+This means that the given index is out of bounds and it causes a crash.

+

Try it out:

+
list.nodeAt(0)!.value    // "Hello"
+list.nodeAt(1)!.value    // "World"
+// list.nodeAt(2)           // crash
+

For fun we can implement a subscript method too:

+
  public subscript(index: Int) -> T {
+    let node = node(atIndex: index)
+    return node.value
+  }
+

Now you can write the following:

+
list[0]   // "Hello"
+list[1]   // "World"
+list[2]   // crash!
+

It crashes on list[2] because there is no node at that index.

+

So far we've written code to add new nodes to the end of the list, but that's slow because you need to find the end of the list first. (It would be fast if we used a tail pointer.) For this reason, if the order of the items in the list doesn't matter, you should insert at the front of the list instead. That's always an O(1) operation.

+

Let's write a method that lets you insert a new node at any index in the list.

+
  public func insert(_ node: Node, atIndex index: Int) {
+   let newNode = node
+   if index == 0 {
+     newNode.next = head                      
+     head?.previous = newNode
+     head = newNode
+   } else {
+     let prev = self.node(atIndex: index-1)
+     let next = prev.next
+
+     newNode.previous = prev
+     newNode.next = prev.next
+     prev.next = newNode
+     next?.previous = newNode
+   }
+}
+

As with node(atIndex :) method, insert(_: at:) method also branches depending on whether the given index is 0 or not.
+First let's look at the former case. Suppose we have the following list and the new node(C).

+
         +---------+     +---------+
+head --->|         |---->|         |-----//----->
+         |    A    |     |    B    |
+ nil <---|         |<----|         |<----//------
+         +---------+     +---------+ 
+             [0]             [1]
+              
+              
+         +---------+ 
+ new --->|         |----> nil
+         |    C    |
+         |         |
+         +---------+
+
+

Now put the new node before the first node. In this way:

+
new.next = head
+head.previous = new
+
+         +---------+            +---------+     +---------+
+ new --->|         |--> head -->|         |---->|         |-----//----->
+         |    C    |            |    A    |     |    B    |
+         |         |<-----------|         |<----|         |<----//------
+         +---------+            +---------+     +---------+ 
+
+

Finally, replace the head with the new node.

+
head = new
+
+         +---------+    +---------+     +---------+
+head --->|         |--->|         |---->|         |-----//----->
+         |    C    |    |    A    |     |    B    |
+ nil <---|         |<---|         |<----|         |<----//------
+         +---------+    +---------+     +---------+ 
+             [0]            [1]             [2]
+
+

However, when the given index is greater than 0, it is necessary to get the node previous and next index and insert between them.
+You can also obtain the previous and next node using node(atIndex:) as follows:

+
         +---------+         +---------+     +---------+    
+head --->|         |---//--->|         |---->|         |----
+         |         |         |    A    |     |    B    |    
+ nil <---|         |---//<---|         |<----|         |<---
+         +---------+         +---------+     +---------+    
+             [0]              [index-1]        [index]      
+                                  ^               ^ 
+                                  |               | 
+                                 prev            next
+
+prev = node(at: index-1)
+next = prev.next
+
+

Now insert new node between the prev and the next.

+
new.prev = prev; prev.next = new  // connect prev and new.
+new.next = next; next.prev = new  // connect new and next.
+
+         +---------+         +---------+     +---------+     +---------+
+head --->|         |---//--->|         |---->|         |---->|         |
+         |         |         |    A    |     |    C    |     |    B    |
+ nil <---|         |---//<---|         |<----|         |<----|         |
+         +---------+         +---------+     +---------+     +---------+
+             [0]              [index-1]        [index]        [index+1]
+                                  ^               ^               ^
+                                  |               |               |
+                                 prev            new             next
+
+

Try it out:

+
list.insert("Swift", atIndex: 1)
+list[0]     // "Hello"
+list[1]     // "Swift"
+list[2]     // "World"
+

Also try adding new nodes to the front and back of the list, to verify that this works properly.

+
+

Note: The node(atIndex:) and insert(_: atIndex:) functions can also be used with a singly linked list because we don't depend on the node's previous pointer to find the previous element.

+
+

What else do we need? Removing nodes, of course! First we'll do removeAll(), which is really simple:

+
  public func removeAll() {
+    head = nil
+  }
+

If you had a tail pointer, you'd set it to nil here too.

+

Next we'll add some functions that let you remove individual nodes. If you already have a reference to the node, then using remove() is the most optimal because you don't need to iterate through the list to find the node first.

+
  public func remove(node: Node) -> T {
+    let prev = node.previous
+    let next = node.next
+
+    if let prev = prev {
+      prev.next = next
+    } else {
+      head = next
+    }
+    next?.previous = prev
+
+    node.previous = nil
+    node.next = nil
+    return node.value
+  }
+

When we take this node out of the list, we break the links to the previous node and the next node. To make the list whole again we must connect the previous node to the next node.

+

Don't forget the head pointer! If this was the first node in the list then head needs to be updated to point to the next node. (Likewise for when you have a tail pointer and this was the last node). Of course, if there are no more nodes left, head should become nil.

+

Try it out:

+
list.remove(list.first!)   // "Hello"
+list.count                     // 2
+list[0]                        // "Swift"
+list[1]                        // "World"
+

If you don't have a reference to the node, you can use removeLast() or removeAt():

+
  public func removeLast() -> T {
+    assert(!isEmpty)
+    return remove(node: last!)
+  }
+
+  public func removeAt(_ index: Int) -> T {
+    let node = nodeAt(index)
+    assert(node != nil)
+    return remove(node: node!)
+  }
+

All these removal functions also return the value from the removed element.

+
list.removeLast()              // "World"
+list.count                     // 1
+list[0]                        // "Swift"
+
+list.removeAt(0)          // "Swift"
+list.count                     // 0
+
+

Note: For a singly linked list, removing the last node is slightly more complicated. You can't just use last to find the end of the list because you also need a reference to the second-to-last node. Instead, use the nodesBeforeAndAfter() helper method. If the list has a tail pointer, then removeLast() is really quick, but you do need to remember to make tail point to the previous node.

+
+

There's a few other fun things we can do with our LinkedList class. It's handy to have some sort of readable debug output:

+
extension LinkedList: CustomStringConvertible {
+  public var description: String {
+    var s = "["
+    var node = head
+    while node != nil {
+      s += "\(node!.value)"
+      node = node!.next
+      if node != nil { s += ", " }
+    }
+    return s + "]"
+  }
+}
+

This will print the list like so:

+
[Hello, Swift, World]
+
+

How about reversing a list, so that the head becomes the tail and vice versa? There is a very fast algorithm for that:

+
  public func reverse() {
+    var node = head
+    tail = node // If you had a tail pointer
+    while let currentNode = node {
+      node = currentNode.next
+      swap(&currentNode.next, &currentNode.previous)
+      head = currentNode
+    }
+  }
+

This loops through the entire list and simply swaps the next and previous pointers of each node. It also moves the head pointer to the very last element. (If you had a tail pointer you'd also need to update it.) You end up with something like this:

+
         +--------+    +--------+    +--------+    +--------+
+tail --->|        |<---|        |<---|        |<---|        |---> nil
+         | node 0 |    | node 1 |    | node 2 |    | node 3 |
+ nil <---|        |--->|        |--->|        |--->|        |<--- head
+         +--------+    +--------+    +--------+    +--------+
+
+

Arrays have map() and filter() functions, and there's no reason why linked lists shouldn't either.

+
  public func map<U>(transform: T -> U) -> LinkedList<U> {
+    let result = LinkedList<U>()
+    var node = head
+    while node != nil {
+      result.append(transform(node!.value))
+      node = node!.next
+    }
+    return result
+  }
+

You can use it like this:

+
let list = LinkedList<String>()
+list.append("Hello")
+list.append("Swifty")
+list.append("Universe")
+
+let m = list.map { s in s.characters.count }
+m  // [5, 6, 8]
+

And here's filter:

+
  public func filter(predicate: T -> Bool) -> LinkedList<T> {
+    let result = LinkedList<T>()
+    var node = head
+    while node != nil {
+      if predicate(node!.value) {
+        result.append(node!.value)
+      }
+      node = node!.next
+    }
+    return result
+  }
+

And a silly example:

+
let f = list.filter { s in s.count > 5 }
+f    // [Universe, Swifty]
+

Exercise for the reader: These implementations of map() and filter() aren't very fast because they append() the new node to the end of the new list. Recall that append is O(n) because it needs to scan through the entire list to find the last node. Can you make this faster? (Hint: keep track of the last node that you added.)

+

An alternative approach

+

The version of LinkedList you've seen so far uses nodes that are classes and therefore have reference semantics. Nothing wrong with that, but that does make them a bit more heavyweight than Swift's other collections such as Array and Dictionary.

+

It is possible to implement a linked list with value semantics using an enum. That would look somewhat like this:

+
enum ListNode<T> {
+  indirect case node(T, next: ListNode<T>)
+  case end
+}
+

The big difference with the enum-based version is that any modification you make to this list will result in a new copy being created because of Swift's value semantics. Whether that's what you want or not depends on the application.

+

[I might fill out this section in more detail if there's a demand for it.]

+

Conforming to the Collection protocol

+

Types that conform to the Sequence protocol, whose elements can be traversed multiple times, nondestructively, and accessed by indexed subscript should conform to the Collection protocol defined in Swift's Standard Library.

+

Doing so grants access to a very large number of properties and operations that are common when dealing collections of data. In addition to this, it lets custom types follow the patterns that are common to Swift developers.

+

In order to conform to this protocol, classes need to provide:
+1 startIndex and endIndex properties.
+2 Subscript access to elements as O(1). Diversions of this time complexity need to be documented.

+
/// The position of the first element in a nonempty collection.
+public var startIndex: Index {
+  get {
+    return LinkedListIndex<T>(node: head, tag: 0)
+  }
+}
+  
+/// The collection's "past the end" position---that is, the position one
+/// greater than the last valid subscript argument.
+/// - Complexity: O(n), where n is the number of elements in the list.
+///   This diverts from the protocol's expectation.
+public var endIndex: Index {
+  get {
+    if let h = self.head {
+      return LinkedListIndex<T>(node: h, tag: count)
+    } else {
+      return LinkedListIndex<T>(node: nil, tag: startIndex.tag)
+    }
+  }
+}
+
public subscript(position: Index) -> T {
+  get {
+    return position.node!.value
+  }
+}
+

Becuase collections are responsible for managing their own indexes, the implementation below keeps a reference to a node in the list. A tag property in the index represents the position of the node in the list.

+
/// Custom index type that contains a reference to the node at index 'tag'
+public struct LinkedListIndex<T> : Comparable
+{
+  fileprivate let node: LinkedList<T>.LinkedListNode<T>?
+  fileprivate let tag: Int
+
+  public static func==<T>(lhs: LinkedListIndex<T>, rhs: LinkedListIndex<T>) -> Bool {
+    return (lhs.tag == rhs.tag)
+  }
+
+  public static func< <T>(lhs: LinkedListIndex<T>, rhs: LinkedListIndex<T>) -> Bool {
+    return (lhs.tag < rhs.tag)
+  }
+}
+

Finally, the linked is is able to calculate the index after a given one with the following implementation.

+
public func index(after idx: Index) -> Index {
+  return LinkedListIndex<T>(node: idx.node?.next, tag: idx.tag+1)
+}
+

Some things to keep in mind

+

Linked lists are flexible but many operations are O(n).

+

When performing operations on a linked list, you always need to be careful to update the relevant next and previous pointers, and possibly also the head and tail pointers. If you mess this up, your list will no longer be correct and your program will likely crash at some point. Be careful!

+

When processing lists, you can often use recursion: process the first element and then recursively call the function again on the rest of the list. You’re done when there is no next element. This is why linked lists are the foundation of functional programming languages such as LISP.

+

Originally written by Matthijs Hollemans for Ray Wenderlich's Swift Algorithm Club

+ + diff --git a/Longest Common Subsequence/index.html b/Longest Common Subsequence/index.html new file mode 100644 index 000000000..5e0272e3f --- /dev/null +++ b/Longest Common Subsequence/index.html @@ -0,0 +1,157 @@ + + + Longest Common Subsequence + + + +

Longest Common Subsequence

+

The Longest Common Subsequence (LCS) of two strings is the longest sequence of characters that appear in the same order in both strings.

+

For example the LCS of "Hello World" and "Bonjour le monde" is "oorld". If you go through both strings from left-to-right, you'll find that the characters o, o, r, l, d appear in both strings in that order.

+

Other possible subsequences are "ed" and "old", but these are all shorter than "oorld".

+
+

Note: This should not be confused with the Longest Common Substring problem, where the characters must form a substring of both strings, i.e they have to be immediate neighbors. With a subsequence, it's OK if the characters are not right next to each other, but they must be in the same order.

+
+

One way to find the LCS of two strings a and b is using dynamic programming and a backtracking strategy.

+

Finding the length of the LCS with dynamic programming

+

First, we want to find the length of the longest common subsequence between strings a and b. We're not looking for the actual subsequence yet, only how long it is.

+

To determine the length of the LCS between all combinations of substrings of a and b, we can use a dynamic programming technique. Dynamic programming basically means that you compute all possibilities and store them inside a look-up table.

+
+

Note: During the following explanation, n is the length of string a, and m is the length of string b.

+
+

To find the lengths of all possible subsequences, we use a helper function, lcsLength(_:). This creates a matrix of size (n+1) by (m+1), where matrix[x][y] is the length of the LCS between the substrings a[0...x-1] and b[0...y-1].

+

Given strings "ABCBX" and "ABDCAB", the output matrix of lcsLength(_:) is the following:

+
|   | Ø | A | B | D | C | A | B |
+| Ø | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+| A | 0 | 1 | 1 | 1 | 1 | 1 | 1 |  
+| B | 0 | 1 | 2 | 2 | 2 | 2 | 2 |
+| C | 0 | 1 | 2 | 2 | 3 | 3 | 3 |
+| B | 0 | 1 | 2 | 2 | 3 | 3 | 4 |
+| X | 0 | 1 | 2 | 2 | 3 | 3 | 4 |
+
+

In this example, if we look at matrix[3][4] we find the value 3. This means the length of the LCS between a[0...2] and b[0...3], or between "ABC" and "ABDC", is 3. That is correct, because these two substrings have the subsequence ABC in common. (Note: the first row and column of the matrix are always filled with zeros.)

+

Here is the source code for lcsLength(_:); this lives in an extension on String:

+
func lcsLength(_ other: String) -> [[Int]] {
+
+  var matrix = [[Int]](repeating: [Int](repeating: 0, count: other.characters.count+1), count: self.characters.count+1)
+
+  for (i, selfChar) in self.characters.enumerated() {
+	for (j, otherChar) in other.characters.enumerated() {
+	  if otherChar == selfChar {
+		// Common char found, add 1 to highest lcs found so far.
+		matrix[i+1][j+1] = matrix[i][j] + 1
+	  } else {
+		// Not a match, propagates highest lcs length found so far.
+		matrix[i+1][j+1] = max(matrix[i][j+1], matrix[i+1][j])
+	  }
+	}
+  }
+
+  return matrix
+}
+

First, this creates a new matrix -- really a 2-dimensional array -- and fills it with zeros. Then it loops through both strings, self and other, and compares their characters in order to fill in the matrix. If two characters match, we increment the length of the subsequence. However, if two characters are different, then we "propagate" the highest LCS length found so far.

+

Let's say the following is the current situation:

+
|   | Ø | A | B | D | C | A | B |
+| Ø | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+| A | 0 | 1 | 1 | 1 | 1 | 1 | 1 |  
+| B | 0 | 1 | 2 | 2 | 2 | 2 | 2 |
+| C | 0 | 1 | 2 | * |   |   |   |
+| B | 0 |   |   |   |   |   |   |
+| X | 0 |   |   |   |   |   |   |
+
+

The * marks the two characters we're currently comparing, C versus D. These characters are not the same, so we propagate the highest length we've seen so far, which is 2:

+
|   | Ø | A | B | D | C | A | B |
+| Ø | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+| A | 0 | 1 | 1 | 1 | 1 | 1 | 1 |  
+| B | 0 | 1 | 2 | 2 | 2 | 2 | 2 |
+| C | 0 | 1 | 2 | 2 | * |   |   |
+| B | 0 |   |   |   |   |   |   |
+| X | 0 |   |   |   |   |   |   |
+
+

Now we compare C with C. These are equal, and we increment the length to 3:

+
|   | Ø | A | B | D | C | A | B |
+| Ø | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+| A | 0 | 1 | 1 | 1 | 1 | 1 | 1 |  
+| B | 0 | 1 | 2 | 2 | 2 | 2 | 2 |
+| C | 0 | 1 | 2 | 2 | 3 | * |   |
+| B | 0 |   |   |   |   |   |   |
+| X | 0 |   |   |   |   |   |   |
+
+

And so on... this is how lcsLength(_:) fills in the entire matrix.

+

Backtracking to find the actual subsequence

+

So far we've calculated the length of every possible subsequence. The length of the longest subsequence is found in the bottom-right corner of matrix, at matrix[n+1][m+1]. In the above example it is 4, so the LCS consists of 4 characters.

+

Having the length of every combination of substrings makes it possible to determine which characters are part of the LCS itself by using a backtracking strategy.

+

Backtracking starts at matrix[n+1][m+1] and walks up and left (in this priority) looking for changes that do not indicate a simple propagation.

+
|   |  Ø|  A|  B|  D|  C|  A|  B|
+| Ø |  0|  0|  0|  0|  0|  0|  0|
+| A |  0|↖ 1|  1|  1|  1|  1|  1|  
+| B |  0|  1|↖ 2|← 2|  2|  2|  2|
+| C |  0|  1|  2|  2|↖ 3|← 3|  3|
+| B |  0|  1|  2|  2|  3|  3|↖ 4|
+| X |  0|  1|  2|  2|  3|  3|↑ 4|
+
+

Each ↖ move indicates a character (in row/column header) that is part of the LCS.

+

If the number on the left and above are different than the number in the current cell, no propagation happened. In that case matrix[i][j] indicates a common char between the strings a and b, so the characters at a[i - 1] and b[j - 1] are part of the LCS that we're looking for.

+

One thing to notice is, as it's running backwards, the LCS is built in reverse order. Before returning, the result is reversed to reflect the actual LCS.

+

Here is the backtracking code:

+
func backtrack(_ matrix: [[Int]]) -> String {
+  var i = self.characters.count
+  var j = other.characters.count
+  
+  var charInSequence = self.endIndex
+  
+  var lcs = String()
+  
+  while i >= 1 && j >= 1 {
+	// Indicates propagation without change: no new char was added to lcs.
+	if matrix[i][j] == matrix[i][j - 1] {
+	  j -= 1
+	}
+	// Indicates propagation without change: no new char was added to lcs.
+	else if matrix[i][j] == matrix[i - 1][j] {
+	  i -= 1
+	  charInSequence = self.index(before: charInSequence)
+	}
+	// Value on the left and above are different than current cell.
+	// This means 1 was added to lcs length.
+	else {
+	  i -= 1
+	  j -= 1
+	  charInSequence = self.index(before: charInSequence)
+	  lcs.append(self[charInSequence])
+	}
+  }
+  
+  return String(lcs.characters.reversed())
+}
+

This backtracks from matrix[n+1][m+1] (bottom-right corner) to matrix[1][1] (top-left corner), looking for characters that are common to both strings. It adds those characters to a new string, lcs.

+

The charInSequence variable is an index into the string given by self. Initially this points to the last character of the string. Each time we decrement i, we also move back charInSequence. When the two characters are found to be equal, we add the character at self[charInSequence] to the new lcs string. (We can't just write self[i] because i may not map to the current position inside the Swift string.)

+

Due to backtracking, characters are added in reverse order, so at the end of the function we call reversed() to put the string in the right order. (Appending new characters to the end of the string and then reversing it once is faster than always inserting the characters at the front of the string.)

+

Putting it all together

+

To find the LCS between two strings, we first call lcsLength(_:) and then backtrack(_:):

+
extension String {
+  public func longestCommonSubsequence(_ other: String) -> String {
+
+    func lcsLength(_ other: String) -> [[Int]] {
+      ...
+    }
+    
+    func backtrack(_ matrix: [[Int]]) -> String {
+      ...
+    }
+
+    return backtrack(lcsLength(other))
+  }
+}
+

To keep everything tidy, the two helper functions are nested inside the main longestCommonSubsequence() function.

+

Here's how you could try it out in a Playground:

+
let a = "ABCBX"
+let b = "ABDCAB"
+a.longestCommonSubsequence(b)   // "ABCB"
+
+let c = "KLMK"
+a.longestCommonSubsequence(c)   // "" (no common subsequence)
+
+"Hello World".longestCommonSubsequence("Bonjour le monde")   // "oorld"
+

Written for Swift Algorithm Club by Pedro Vereza

+ + diff --git a/Merge Sort/index.html b/Merge Sort/index.html new file mode 100644 index 000000000..8b0392f40 --- /dev/null +++ b/Merge Sort/index.html @@ -0,0 +1,232 @@ + + + Merge Sort + + + +

Merge Sort

+
+

This topic has been tutorialized here

+
+

Goal: Sort an array from low to high (or high to low)

+

Invented in 1945 by John von Neumann, merge-sort is an efficient algorithm with a best, worst, and average time complexity of O(n log n).

+

The merge-sort algorithm uses the divide and conquer approach which is to divide a big problem into smaller problems and solve them. I think of the merge-sort algorithm as split first and merge after.

+

Assume you need to sort an array of n numbers in the right order. The merge-sort algorithm works as follows:

+ +

An example

+

Splitting

+

Assume you are given an array of n numbers as[2, 1, 5, 4, 9]. This is an unsorted pile. The goal is to keep splitting the pile until you cannot split anymore.

+

First, split the array into two halves: [2, 1] and [5, 4, 9]. Can you keep splitting them? Yes, you can!

+

Focus on the left pile. Split[2, 1] into [2] and [1]. Can you keep splitting them? No. Time to check the other pile.

+

Split [5, 4, 9] into [5] and [4, 9]. Unsurprisingly, [5] cannot be split anymore, but [4, 9] can be split into [4] and [9].

+

The splitting process ends with the following piles: [2] [1] [5] [4] [9]. Notice that each pile consists of just one element.

+

Merging

+

Now that you have split the array, you should merge the piles together while sorting them. Remember, the idea is to solve many small problems rather than a big one. For each merge iteration, you must be concerned at merging one pile with another.

+

Given the piles [2] [1] [5] [4] [9], the first pass will result in [1, 2] and [4, 5] and [9]. Since [9] is the odd one out, you cannot merge it with anything during this pass.

+

The next pass will merge [1, 2] and [4, 5] together. This results in [1, 2, 4, 5], with the [9] left out again because it is the odd one out.

+

You are left with only two piles [1, 2, 4, 5] and [9], finally gets its chance to merge, resulting in the sorted array as [1, 2, 4, 5, 9].

+

Top-down implementation

+

Here's what merge sort may look like in Swift:

+
func mergeSort(_ array: [Int]) -> [Int] {
+  guard array.count > 1 else { return array }    // 1
+
+  let middleIndex = array.count / 2              // 2
+
+  let leftArray = mergeSort(Array(array[0..<middleIndex]))             // 3
+
+  let rightArray = mergeSort(Array(array[middleIndex..<array.count]))  // 4
+
+  return merge(leftPile: leftArray, rightPile: rightArray)             // 5
+}
+

A step-by-step explanation of how the code works:

+
    +
  1. +

    If the array is empty or contains a single element, there is no way to split it into smaller pieces. You must just return the array.

    +
    2. +
  2. +

    Find the middle index.

    +
    3. +
  3. +

    Using the middle index from the previous step, recursively split the left side of the array.

    +
    4. +
  4. +

    Also, recursively split the right side of the array.

    +
    5. +
  5. +

    Finally, merge all the values together, making sure that it is always sorted.

    +
    6. +
+

Here's the merging algorithm:

+
func merge(leftPile: [Int], rightPile: [Int]) -> [Int] {
+  // 1
+  var leftIndex = 0
+  var rightIndex = 0
+
+  // 2
+  var orderedPile = [Int]()
+  orderedPile.reserveCapacity(leftPile.count + rightPile.count)
+
+  // 3
+  while leftIndex < leftPile.count && rightIndex < rightPile.count {
+    if leftPile[leftIndex] < rightPile[rightIndex] {
+      orderedPile.append(leftPile[leftIndex])
+      leftIndex += 1
+    } else if leftPile[leftIndex] > rightPile[rightIndex] {
+      orderedPile.append(rightPile[rightIndex])
+      rightIndex += 1
+    } else {
+      orderedPile.append(leftPile[leftIndex])
+      leftIndex += 1
+      orderedPile.append(rightPile[rightIndex])
+      rightIndex += 1
+    }
+  }
+
+  // 4
+  while leftIndex < leftPile.count {
+    orderedPile.append(leftPile[leftIndex])
+    leftIndex += 1
+  }
+
+  while rightIndex < rightPile.count {
+    orderedPile.append(rightPile[rightIndex])
+    rightIndex += 1
+  }
+
+  return orderedPile
+}
+

This method may look scary, but it is quite straightforward:

+
    +
  1. +

    You need two indexes to keep track of your progress for the two arrays while merging.

    +
    2. +
  2. +

    This is the merged array. It is empty right now, but you will build it up in subsequent steps by appending elements from the other arrays. Since you already know number of elements that will end up in this array, you reserve capacity to avoid reallocation overhead later.

    +
    3. +
  3. +

    This while-loop will compare the elements from the left and right sides and append them into the orderedPile while making sure that the result stays in order.

    +
    4. +
  4. +

    If control exits from the previous while-loop, it means that either the leftPile or the rightPile has its contents completely merged into the orderedPile. At this point, you no longer need to do comparisons. Just append the rest of the contents of the other array until there is no more to append.

    +
    5. +
+

As an example of how merge() works, suppose that we have the following piles: leftPile = [1, 7, 8] and rightPile = [3, 6, 9]. Note that each of these piles is individually sorted already -- that is always true with merge sort. These are merged into one larger sorted pile in the following steps:

+
leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ ]
+  l              r
+
+

The left index, here represented as l, points at the first item from the left pile, 1. The right index, r, points at 3. Therefore, the first item we add to orderedPile is 1. We also move the left index l to the next item.

+
leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ 1 ]
+  -->l           r
+
+

Now l points at 7 but r is still at 3. We add the smallest item to the ordered pile, so that is 3. The situation is now:

+
leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ 1, 3 ]
+     l           -->r
+
+

This process repeats. At each step, we pick the smallest item from either the leftPile or the rightPile and add the item into the orderedPile:

+
leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ 1, 3, 6 ]
+     l              -->r
+
+leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ 1, 3, 6, 7 ]
+     -->l              r
+
+leftPile       rightPile       orderedPile
+[ 1, 7, 8 ]    [ 3, 6, 9 ]     [ 1, 3, 6, 7, 8 ]
+        -->l           r
+
+

Now, there are no more items in the left pile. We simply add the remaining items from the right pile, and we are done. The merged pile is [ 1, 3, 6, 7, 8, 9 ].

+

Notice that, this algorithm is very simple: it moves from left-to-right through the two piles and at every step picks the smallest item. This works because we guarantee that each of the piles is already sorted.

+

Bottom-up implementation

+

The implementation of the merge-sort algorithm you have seen so far is called the "top-down" approach because it first splits the array into smaller piles and then merges them. When sorting an array (as opposed to, say, a linked list) you can actually skip the splitting step and immediately start merging the individual array elements. This is called the "bottom-up" approach.

+

Time to step up the game a little. :-) Here is a complete bottom-up implementation in Swift:

+
func mergeSortBottomUp<T>(_ a: [T], _ isOrderedBefore: (T, T) -> Bool) -> [T] {
+  let n = a.count
+
+  var z = [a, a]      // 1
+  var d = 0
+
+  var width = 1
+  while width < n {   // 2
+
+    var i = 0
+    while i < n {     // 3
+
+      var j = i
+      var l = i
+      var r = i + width
+
+      let lmax = min(l + width, n)
+      let rmax = min(r + width, n)
+
+      while l < lmax && r < rmax {                // 4
+        if isOrderedBefore(z[d][l], z[d][r]) {
+          z[1 - d][j] = z[d][l]
+          l += 1
+        } else {
+          z[1 - d][j] = z[d][r]
+          r += 1
+        }
+        j += 1
+      }
+      while l < lmax {
+        z[1 - d][j] = z[d][l]
+        j += 1
+        l += 1
+      }
+      while r < rmax {
+        z[1 - d][j] = z[d][r]
+        j += 1
+        r += 1
+      }
+
+      i += width*2
+    }
+
+    width *= 2
+    d = 1 - d      // 5
+  }
+  return z[d]
+}
+

It looks a lot more intimidating than the top-down version, but notice that the main body includes the same three while loops from merge().

+

Notable points:

+
    +
  1. +

    The Merge-sort algorithm needs a temporary working array because you cannot merge the left and right piles and at the same time overwrite their contents. Because allocating a new array for each merge is wasteful, we are using two working arrays, and we will switch between them using the value of d, which is either 0 or 1. The array z[d] is used for reading, and z[1 - d] is used for writing. This is called double-buffering.

    +
    2. +
  2. +

    Conceptually, the bottom-up version works the same way as the top-down version. First, it merges small piles of one element each, then it merges piles of two elements each, then piles of four elements each, and so on. The size of the pile is given by width. Initially, width is 1 but at the end of each loop iteration, we multiply it by two, so this outer loop determines the size of the piles being merged, and the subarrays to merge become larger in each step.

    +
    3. +
  3. +

    The inner loop steps through the piles and merges each pair of piles into a larger one. The result is written in the array given by z[1 - d].

    +
    4. +
  4. +

    This is the same logic as in the top-down version. The main difference is that we're using double-buffering, so values are read from z[d] and written into z[1 - d]. It also uses an isOrderedBefore function to compare the elements rather than just <, so this merge-sort algorithm is generic, and you can use it to sort any kind of object you want.

    +
    5. +
  5. +

    At this point, the piles of size width from array z[d] have been merged into larger piles of size width * 2 in array z[1 - d]. Here, we swap the active array, so that in the next step we'll read from the new piles we have just created.

    +
    6. +
+

This function is generic, so you can use it to sort any type you desire, as long as you provide a proper isOrderedBefore closure to compare the elements.

+

Example of how to use it:

+
let array = [2, 1, 5, 4, 9]
+mergeSortBottomUp(array, <)   // [1, 2, 4, 5, 9]
+

Performance

+

The speed of the merge-sort algorithm is dependent on the size of the array it needs to sort. The larger the array, the more work it needs to do.

+

Whether or not the initial array is sorted already does not affect the speed of the merge-sort algorithm since you will be doing the same amount splits and comparisons regardless of the initial order of the elements.

+

Therefore, the time complexity for the best, worst, and average case will always be O(n log n).

+

A disadvantage of the merge-sort algorithm is that it needs a temporary "working" array equal in size to the array being sorted. It is not an in-place sort, unlike for example quicksort.

+

Most implementations of the merge-sort algorithm produce a stable sort. This means that array elements that have identical sort keys will stay in the same order relative to each other after sorting. This is not important for simple values such as numbers or strings, but it can be an issue when sorting more complex objects.

+

See also

+

Merge sort on Wikipedia

+

Written by Kelvin Lau. Additions by Matthijs Hollemans.

+ + diff --git a/Miller-Rabin Primality Test/index.html b/Miller-Rabin Primality Test/index.html new file mode 100644 index 000000000..cb0cf34ab --- /dev/null +++ b/Miller-Rabin Primality Test/index.html @@ -0,0 +1,34 @@ + + + Miller-Rabin Primality Test + + + +

Miller-Rabin Primality Test

+

In 1976, Gray Miller introduced an algorithm, through his ph.d thesis1, which determines a primality of the given number. The original algorithm was deterministic under the Extended Reimann Hypothesis, which is yet to be proven. After four years, Michael O. Rabin improved the algorithm2 by using probabilistic approach and it no longer assumes the unproven hypothesis.

+

Probabilistic

+

The result of the test is simply a boolean. However, true does not implicate the number is prime. In fact, it means the number is probably prime. But false does mean the number is composite.

+

In order to increase the accuracy of the test, it needs to be iterated few times. If it returns true in every single iteration, then we can say with confidence that the number is pro......bably prime.

+

Algorithm

+

Let n be the given number, and write n-1 as 2^s·d, where d is odd. And choose a random number a within the range from 2 to n - 1.

+

Now make a sequence, in modulo n, as following:

+
+

a^d, a^(2·d), a^(4·d), ... , a^((2^(s-1))·d), a^((2^s)·d) = a^(n-1)

+
+

And we say the number n passes the test, probably prime, if 1) a^d is congruence to 1 in modulo n, or 2) a^((2^k)·d) is congruence to -1 for some k = 1, 2, ..., s-1.

+

Pseudo Code

+

The following pseudo code is excerpted from Wikipedia3:

+

Image of Pseudocode

+

Usage

+
checkWithMillerRabin(7)                      // test if 7 is prime. (default iteration = 1)
+checkWithMillerRabin(7, accuracy: 10)       // test if 7 is prime && iterate 10 times.
+

Reference

+
    +
  1. G. L. Miller, "Riemann's Hypothesis and Tests for Primality". J. Comput. System Sci. 13 (1976), 300-317.
    2. +
  2. M. O. Rabin, "Probabilistic algorithm for testing primality". Journal of Number Theory. 12 (1980), 128-138.
    3. +
  3. Miller–Rabin primality test - Wikipedia, the free encyclopedia
    4. +
+

Written for Swift Algorithm Club by Sahn Cha, @scha00
+Code updated by Simon C. Krüger.

+ + diff --git a/Minimum Edit Distance/index.html b/Minimum Edit Distance/index.html new file mode 100644 index 000000000..b2557c023 --- /dev/null +++ b/Minimum Edit Distance/index.html @@ -0,0 +1,50 @@ + + + Minimum Edit Distance + + + +

Minimum Edit Distance

+

The minimum edit distance is a possibility to measure the similarity of two strings w and u by counting costs of operations which are necessary to transform w into u (or vice versa).

+

Algorithm using Levenshtein distance

+

A common distance measure is given by the Levenshtein distance, which allows the following three transformation operations:

+ +

When transforming a string by a sequence of operations, the costs of the single operations are added to obtain the (minimal) edit distance. For example, the string Door can be transformed by the operations o→l, r→l, ε→s to the string Dolls, which results in a minimum edit distance of 3.

+

To avoid exponential time complexity, the minimum edit distance of two strings in the usual is computed using dynamic programming. For this in a matrix

+
var matrix = [[Int]](repeating: [Int](repeating: 0, count: n + 1), count: m + 1)
+

already computed minimal edit distances of prefixes of w and u (of length m and n, respectively) are used to fill the matrix. In a first step the matrix is initialized by filling the first row and the first column as follows:

+
// initialize matrix
+for index in 1...m {
+    // the distance of any first string to an empty second string
+    matrix[index][0] = index
+}
+
+for index in 1...n {
+    // the distance of any second string to an empty first string
+    matrix[0][index] = index
+}
+

Then in each cell the minimum of the cost of insertion, deletion, or substitution added to the already computed costs in the corresponding cells is chosen. In this way the matrix is filled iteratively:

+
// compute Levenshtein distance
+for (i, selfChar) in self.enumerated() {
+    for (j, otherChar) in other.enumerated() {
+        if otherChar == selfChar {
+            // substitution of equal symbols with cost 0
+            matrix[i + 1][j + 1] = matrix[i][j]
+        } else {
+            // minimum of the cost of insertion, deletion, or substitution 
+            // added to the already computed costs in the corresponding cells
+            matrix[i + 1][j + 1] = Swift.min(matrix[i][j] + 1, matrix[i + 1][j] + 1, matrix[i][j + 1] + 1)
+        } 
+    }
+}
+

After applying this algorithm, the minimal edit distance can be read from the rightmost bottom cell and is returned.

+
return matrix[m][n]
+

This algorithm has a time complexity of Θ(mn).

+

TODO: Other distance measures.

+

Written for Swift Algorithm Club by Luisa Herrmann

+ + diff --git a/Minimum Spanning Tree (Unweighted)/index.html b/Minimum Spanning Tree (Unweighted)/index.html new file mode 100644 index 000000000..9f4ad20e8 --- /dev/null +++ b/Minimum Spanning Tree (Unweighted)/index.html @@ -0,0 +1,139 @@ + + + Minimum Spanning Tree (Unweighted) + + + +

Minimum Spanning Tree (Unweighted Graph)

+

A minimum spanning tree describes a path that contains the smallest number of edges that are needed to visit every node in the graph.

+

Take a look at the following graph:

+

Graph

+

If we start from node a and want to visit every other node, then what is the most efficient path to do that? We can calculate this with the minimum spanning tree algorithm.

+

Here is the minimum spanning tree for the graph. It is represented by the bold edges:

+

Minimum spanning tree

+

Drawn as a more conventional tree it looks like this:

+

An actual tree

+

To calculate the minimum spanning tree on an unweighted graph, we can use the breadth-first search algorithm. Breadth-first search starts at a source node and traverses the graph by exploring the immediate neighbor nodes first, before moving to the next level neighbors. If we tweak this algorithm by selectively removing edges, then it can convert the graph into the minimum spanning tree.

+

Let's step through the example. We start with the source node a, add it to a queue and mark it as visited.

+
queue.enqueue(a)
+a.visited = true
+

The queue is now [ a ]. As is usual with breadth-first search, we dequeue the node at the front of the queue, a, and enqueue its immediate neighbor nodes b and h. We mark them as visited too.

+
queue.dequeue()   // a
+queue.enqueue(b)
+b.visited = true
+queue.enqueue(h)
+h.visited = true
+

The queue is now [ b, h ]. Dequeue b and enqueue the neighbor node c. Mark it as visited. Remove the edge from b to h because h has already been visited.

+
queue.dequeue()   // b
+queue.enqueue(c)
+c.visited = true
+b.removeEdgeTo(h)
+

The queue is now [ h, c ]. Dequeue h and enqueue the neighbor nodes g and i, and mark them as visited.

+
queue.dequeue()   // h
+queue.enqueue(g)
+g.visited = true
+queue.enqueue(i)
+i.visited = true
+

The queue is now [ c, g, i ]. Dequeue c and enqueue the neighbor nodes d and f, and mark them as visited. Remove the edge between c and i because i has already been visited.

+
queue.dequeue()   // c
+queue.enqueue(d)
+d.visited = true
+queue.enqueue(f)
+f.visited = true
+c.removeEdgeTo(i)
+

The queue is now [ g, i, d, f ]. Dequeue g. All of its neighbors have been discovered already, so there is nothing to enqueue. Remove the edges from g to f, as well as g to i, because f and i have already been discovered.

+
queue.dequeue()   // g
+g.removeEdgeTo(f)
+g.removeEdgeTo(i)
+

The queue is now [ i, d, f ]. Dequeue i. Nothing else to do for this node.

+
queue.dequeue()   // i
+

The queue is now [ d, f ]. Dequeue d and enqueue the neighbor node e. Mark it as visited. Remove the edge from d to f because f has already been visited.

+
queue.dequeue()   // d
+queue.enqueue(e)
+e.visited = true
+d.removeEdgeTo(f)
+

The queue is now [ f, e ]. Dequeue f. Remove the edge between f and e because e has already been visited.

+
queue.dequeue()   // f
+f.removeEdgeTo(e)
+

The queue is now [ e ]. Dequeue e.

+
queue.dequeue()   // e
+

The queue is empty, which means the minimum spanning tree has been computed.

+

Here's the code:

+
func breadthFirstSearchMinimumSpanningTree(graph: Graph, source: Node) -> Graph {
+  let minimumSpanningTree = graph.duplicate()
+
+  var queue = Queue<Node>()
+  let sourceInMinimumSpanningTree = minimumSpanningTree.findNodeWithLabel(source.label)
+  queue.enqueue(sourceInMinimumSpanningTree)
+  sourceInMinimumSpanningTree.visited = true
+
+  while let current = queue.dequeue() {
+    for edge in current.neighbors {
+      let neighborNode = edge.neighbor
+      if !neighborNode.visited {
+        neighborNode.visited = true
+        queue.enqueue(neighborNode)
+      } else {
+        current.remove(edge)
+      }
+    }
+  }
+
+  return minimumSpanningTree
+}
+

This function returns a new Graph object that describes just the minimum spanning tree. In the figure, that would be the graph containing just the bold edges.

+

Put this code in a playground and test it like so:

+
let graph = Graph()
+
+let nodeA = graph.addNode("a")
+let nodeB = graph.addNode("b")
+let nodeC = graph.addNode("c")
+let nodeD = graph.addNode("d")
+let nodeE = graph.addNode("e")
+let nodeF = graph.addNode("f")
+let nodeG = graph.addNode("g")
+let nodeH = graph.addNode("h")
+let nodeI = graph.addNode("i")
+
+graph.addEdge(nodeA, neighbor: nodeB)
+graph.addEdge(nodeA, neighbor: nodeH)
+graph.addEdge(nodeB, neighbor: nodeA)
+graph.addEdge(nodeB, neighbor: nodeC)
+graph.addEdge(nodeB, neighbor: nodeH)
+graph.addEdge(nodeC, neighbor: nodeB)
+graph.addEdge(nodeC, neighbor: nodeD)
+graph.addEdge(nodeC, neighbor: nodeF)
+graph.addEdge(nodeC, neighbor: nodeI)
+graph.addEdge(nodeD, neighbor: nodeC)
+graph.addEdge(nodeD, neighbor: nodeE)
+graph.addEdge(nodeD, neighbor: nodeF)
+graph.addEdge(nodeE, neighbor: nodeD)
+graph.addEdge(nodeE, neighbor: nodeF)
+graph.addEdge(nodeF, neighbor: nodeC)
+graph.addEdge(nodeF, neighbor: nodeD)
+graph.addEdge(nodeF, neighbor: nodeE)
+graph.addEdge(nodeF, neighbor: nodeG)
+graph.addEdge(nodeG, neighbor: nodeF)
+graph.addEdge(nodeG, neighbor: nodeH)
+graph.addEdge(nodeG, neighbor: nodeI)
+graph.addEdge(nodeH, neighbor: nodeA)
+graph.addEdge(nodeH, neighbor: nodeB)
+graph.addEdge(nodeH, neighbor: nodeG)
+graph.addEdge(nodeH, neighbor: nodeI)
+graph.addEdge(nodeI, neighbor: nodeC)
+graph.addEdge(nodeI, neighbor: nodeG)
+graph.addEdge(nodeI, neighbor: nodeH)
+
+let minimumSpanningTree = breadthFirstSearchMinimumSpanningTree(graph, source: nodeA)
+
+print(minimumSpanningTree) // [node: a edges: ["b", "h"]]
+                           // [node: b edges: ["c"]]
+                           // [node: c edges: ["d", "f"]]
+                           // [node: d edges: ["e"]]
+                           // [node: h edges: ["g", "i"]]
+
+

Note: On an unweighed graph, any spanning tree is always a minimal spanning tree. This means you can also use a depth-first search to find the minimum spanning tree.

+
+

Written by Chris Pilcher and Matthijs Hollemans

+ + diff --git a/Minimum Spanning Tree/index.html b/Minimum Spanning Tree/index.html new file mode 100644 index 000000000..d3285c9ea --- /dev/null +++ b/Minimum Spanning Tree/index.html @@ -0,0 +1,88 @@ + + + Minimum Spanning Tree + + + +

Minimum Spanning Tree (Weighted Graph)

+
+

This topic has been tutorialized here

+
+

A minimum spanning tree (MST) of a connected undirected weighted graph has a subset of the edges from the original graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. There can be more than one MSTs of a graph.

+

There are two popular algorithms to calculate MST of a graph - Kruskal's algorithm and Prim's algorithm. Both algorithms have a total time complexity of O(ElogE) where E is the number of edges from the original graph.

+

Kruskal's Algorithm

+

Sort the edges base on weight. Greedily select the smallest one each time and add into the MST as long as it doesn't form a cycle.
+Kruskal's algoritm uses Union Find data structure to check whether any additional edge causes a cycle. The logic is to put all connected vertices into the same set (in Union Find's concept). If the two vertices from a new edge do not belong to the same set, then it's safe to add that edge into the MST.

+

The following graph demonstrates the steps:

+

Graph

+

Preparation

+
// Initialize the values to be returned and Union Find data structure.
+var cost: Int = 0
+var tree = Graph<T>()
+var unionFind = UnionFind<T>()
+for vertex in graph.vertices {
+
+// Initially all vertices are disconnected.
+// Each of them belongs to it's individual set.
+  unionFind.addSetWith(vertex)
+}
+

Sort the edges

+
let sortedEdgeListByWeight = graph.edgeList.sorted(by: { $0.weight < $1.weight })
+

Take one edge at a time and try to insert it into the MST.

+
for edge in sortedEdgeListByWeight {
+  let v1 = edge.vertex1
+  let v2 = edge.vertex2 
+  
+  // Same set means the two vertices of this edge were already connected in the MST.
+  // Adding this one will cause a cycle.
+  if !unionFind.inSameSet(v1, and: v2) {
+    // Add the edge into the MST and update the final cost.
+    cost += edge.weight
+    tree.addEdge(edge)
+    
+    // Put the two vertices into the same set.
+    unionFind.unionSetsContaining(v1, and: v2)
+  }
+}
+

Prim's Algorithm

+

Prim's algorithm doesn't pre-sort all edges. Instead, it uses a Priority Queue to maintain a running sorted next-possile vertices.
+Starting from one vertex, loop through all unvisited neighbours and enqueue a pair of values for each neighbour - the vertex and the weight of edge connecting current vertex to the neighbour. Each time it greedily select the top of the priority queue (the one with least weight value) and add the edge into the final MST if the enqueued neighbour hasn't been already visited.

+

The following graph demonstrates the steps:

+

Graph

+

Preparation

+
// Initialize the values to be returned and Priority Queue data structure.
+var cost: Int = 0
+var tree = Graph<T>()
+var visited = Set<T>()
+
+// In addition to the (neighbour vertex, weight) pair, parent is added for the purpose of printing out the MST later.
+// parent is basically current vertex. aka. the previous vertex before neigbour vertex gets visited.
+var priorityQueue = PriorityQueue<(vertex: T, weight: Int, parent: T?)>(sort: { $0.weight < $1.weight })
+

Start from any vertex

+
priorityQueue.enqueue((vertex: graph.vertices.first!, weight: 0, parent: nil))
+
// Take from the top of the priority queue ensures getting the least weight edge.
+while let head = priorityQueue.dequeue() {
+  let vertex = head.vertex
+  if visited.contains(vertex) {
+    continue
+  }
+
+  // If the vertex hasn't been visited before, its edge (parent-vertex) is selected for MST.
+  visited.insert(vertex)
+  cost += head.weight
+  if let prev = head.parent { // The first vertex doesn't have a parent.
+    tree.addEdge(vertex1: prev, vertex2: vertex, weight: head.weight)
+  }
+
+  // Add all unvisted neighbours into the priority queue.
+  if let neighbours = graph.adjList[vertex] {
+    for neighbour in neighbours {
+      let nextVertex = neighbour.vertex
+      if !visited.contains(nextVertex) {
+        priorityQueue.enqueue((vertex: nextVertex, weight: neighbour.weight, parent: vertex))
+      }
+    }
+  }
+}
+ + diff --git a/MinimumCoinChange/index.html b/MinimumCoinChange/index.html new file mode 100644 index 000000000..a7275113b --- /dev/null +++ b/MinimumCoinChange/index.html @@ -0,0 +1,31 @@ + + + MinimumCoinChange + + + +

Minimum Coin Change

+

Minimum Coin Change problem algorithm implemented in Swift comparing dynamic programming algorithm design to traditional greedy approach.

+

Written for Swift Algorithm Club by Jacopo Mangiavacchi

+

Coins

+

Introduction

+

In the traditional coin change problem you have to find all the different ways to change some given money in a particular amount of coins using a given amount of set of coins (i.e. 1 cent, 2 cents, 5 cents, 10 cents etc.).

+

For example using Euro cents the total of 4 cents value of money can be changed in these possible ways:

+ +

The minimum coin change problem is a variation of the generic coin change problem where you need to find the best option for changing the money returning the less number of coins.

+

For example using Euro cents the best possible change for 4 cents are two 2 cent coins with a total of two coins.

+

Greedy Solution

+

A simple approach for implementing the Minimum Coin Change algorithm in a very efficient way is to start subtracting from the input value the greater possible coin value from the given amount of set of coins available and iterate subtracting the next greater possible coin value on the resulting difference.

+

For example from the total of 4 Euro cents of the example above you can subtract initially 2 cents as the other biggest coins value (from 5 cents to above) are to bigger for the current 4 Euro cent value. Once used the first 2 cents coin you iterate again with the same logic for the rest of 2 cents and select another 2 cents coin and finally return the two 2 cents coins as the best change.

+

Most of the time the result for this greedy approach is optimal but for some set of coins the result will not be the optimal.

+

Indeed, if we use the a set of these three different coins set with values 1, 3 and 4 and execute this greedy algorithm for asking the best change for the value 6 we will get one coin of 4 and two coins of 1 instead of two coins of 3.

+

Dynamic Programming Solution

+

A classic dynamic programming strategy will iterate selecting in order a possible coin from the given amount of set of coins and finding using recursive calls the minimum coin change on the difference from the passed value and the selected coin. For any interaction the algorithm select from all possible combinations the one with the less number of coins used.

+

The dynamic programming approach will always select the optimal change but it will require a number of steps that is at least quadratic in the goal amount to change.

+

In this Swift implementation in order to optimize the overall performance we use an internal data structure for caching the result for best minimum coin change for previous values.

+ + diff --git a/Monty Hall Problem/index.html b/Monty Hall Problem/index.html new file mode 100644 index 000000000..b12d5c146 --- /dev/null +++ b/Monty Hall Problem/index.html @@ -0,0 +1,55 @@ + + + Monty Hall Problem + + + +

The Monty Hall Problem

+

Congrats! You've reached the final round of the popular Monty Hall game show. Monty, the show host, gives you the choice between 3 doors. Behind one of the doors is a prize (a new car? a trip to Hawaii? a microwave oven?), the other two are empty.

+

After you make your choice, Monty decides to make things a bit more interesting and opens one of the two doors that you didn't pick. Of course, the one he opens is empty. There are now two doors left, behind one of which is the coveted prize.

+

Now Monty gives you the opportunity to change your mind. Should you stick with your original choice, should you pick the other door, or doesn't it matter?

+

You'd think that changing your answer wouldn't improve your chances... but it does!

+

This is a very nonintuitive result. Maybe you have trouble believing this is true. Don't worry, when this problem was first proposed many professional mathematicians didn't believe it either, so you're in good company.

+

There's a simple way to verify this claim: we can write a program to test it out! We should be able to show who wins more often by playing the game a large number of times.

+

Here's the code (see the playground for the full thing). First, we randomly choose the door that has the prize:

+
  let prizeDoor = random(3)
+

We also randomly pick the choice of the player:

+
  let chooseDoor = random(3)
+

Next, Monty opens one of the empty doors. Obviously, he won't choose the door that the player chose or the one with the prize.

+
  var openDoor = -1
+  repeat {
+    openDoor = random(3)
+  } while openDoor == prizeDoor || openDoor == chooseDoor
+

There are only two closed doors left, one of which has the prize. What happens when the player changes his mind and picks the other door?

+
  var changeMind = -1
+  repeat {
+    changeMind = random(3)
+  } while changeMind == openDoor || changeMind == chooseDoor
+

Now we see which choice was the right one:

+
  if chooseDoor == prizeDoor {
+    winOriginalChoice += 1
+  }
+  if changeMind == prizeDoor {
+    winChangedMind += 1
+  }
+

If the prize is behind the player's original door choice, we increment winOriginalChoice. If the prize is behind the other door, then the player would have won if he changed his mind, and so we increment winChangedMind.

+

And that's all there is to it.

+

If you run the above code 1000 times or so, you'll find that the probability of choosing the prize without changing your mind is about 33%. But if you do change your mind, the probability of winning is 67% -- that is twice as large!

+

Try it out in the playground if you still don't believe it. ;-)

+

Here's why: When you first make a choice, your chances of picking the prize are 1 out of 3, or 33%

+

After Monty opens one of the doors, this gives you new information. However, it doesn't change the probability of your original choice being the winner. That chance remains 33% because you made that choice when you didn't know yet what was behind this open door.

+

Since probabilities always need to add up to 100%, the chance that the prize is behind the other door is now 100 - 33 = 67%. So, as strange as it may sound, you're better off switching doors!

+

This is hard to wrap your head around, but easily shown using a simulation that runs a significant number of times. Probability is weird.

+

By the way, you can simplify the code to this:

+
  let prizeDoor = random(3)
+  let chooseDoor = random(3)
+  if chooseDoor == prizeDoor {
+    winOriginalChoice += 1
+  } else {
+    winChangedMind += 1
+  }
+

Now it's no longer a simulation but the logic is equivalent. You can clearly see that the chooseDoor only wins 1/3rd of the time -- because it's a random number between 1 and 3 -- so changing your mind must win the other 2/3rds of the time.

+

Monty Hall Problem on Wikipedia

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Multiset/index.html b/Multiset/index.html new file mode 100644 index 000000000..3efef2976 --- /dev/null +++ b/Multiset/index.html @@ -0,0 +1,73 @@ + + + Multiset + + + +

Multiset

+

A multiset (also known as a bag) is a data structure similar to a regular set, but it can store multiple instances of the same element.

+

For example, if I added the elements 1, 2, 2 to a regular set, the set would only contain two items, since adding 2 a second time has no effect.

+
var set = Set<Int>()
+set.add(1) // set is now [1]
+set.add(2) // set is now [1, 2]
+set.add(2) // set is still [1, 2]
+
+

By comparison, after adding the elements 1, 2, 2 to a multiset, it would contain three items.

+
var set = Multiset<Int>()
+set.add(1) // set is now [1]
+set.add(2) // set is now [1, 2]
+set.add(2) // set is now [1, 2, 2]
+
+

You might be thinking that this looks an awful lot like an array. So why would you use a multiset? Let's consider the differences between the two…

+ +

Typical operations on a multiset are:

+ +

One real-world use of multisets is to determine whether one string is a partial anagram of another. For example, the word "cacti" is a partial anagrams of "tactical". (In other words, I can rearrange the letters of "tactical" to make "cacti", with some letters left over.)

+
var cacti = Multiset<Character>("cacti")
+var tactical = Multiset<Character>("tactical")
+cacti.isSubSet(of: tactical) // true!
+

Implementation

+

Under the hood, this implementation of Multiset uses a dictionary to store a mapping of elements to the number of times they've been added.

+

Here's the essence of it:

+
public struct Multiset<Element: Hashable> {
+  private var storage: [Element: UInt] = [:]
+  
+  public init() {}
+

And here's how you'd use this class to create a multiset of strings:

+
var set = Multiset<String>()
+

Adding an element is a case of incrementing the counter for that element, or setting it to 1 if it doesn't already exist:

+
public mutating func add (_ elem: Element) {
+  storage[elem, default: 0] += 1
+}
+

Here's how you'd use this method to add to the set we created earlier:

+
set.add("foo")
+set.add("foo") 
+set.allItems // returns ["foo", "foo"]
+

Our set now contains two elements, both the string "foo".

+

Removing an element works much the same way as adding; decrement the counter for the element, or remove it from the underlying dictionary if its value is 1 before removal.

+
public mutating func remove (_ elem: Element) {
+  if let currentCount = storage[elem] {
+    if currentCount > 1 {
+      storage[elem] = currentCount - 1
+    } else {
+      storage.removeValue(forKey: elem)
+    }
+  }
+}
+

Getting the count for an item is simple: we just return the value for the given item in the internal dictionary.

+
public func count(for key: Element) -> UInt {
+  return storage[key] ?? 0
+}
+

Written for the Swift Algorithm Club by Simon Whitaker

+ + diff --git a/Myers Difference Algorithm/index.html b/Myers Difference Algorithm/index.html new file mode 100644 index 000000000..909a7dc57 --- /dev/null +++ b/Myers Difference Algorithm/index.html @@ -0,0 +1,140 @@ + + + Myers Difference Algorithm + + + +

Myers Difference Algorithm

+

Myers Difference Algorithm(MDA) is an algorithm that finds a longest common subsequence(LCS) or shortest edit scripts(SES) of two sequences. The common subsequence of two sequences is the sequence of elements that appear in the same order in both sequences. For example, let's assume you have two arrays:

+
let firstArray = [1, 2, 3]
+let secondArray = [2, 3, 4]
+
+

The common subsequences of these two arrays are [2], and [2, 3]. The longest common sequence in this case is [2, 3]. MDA can accomplish this in O(ND) time, where N is the sum of the lengths of the two sequences.

+

Finding the length of the Longest Common Subsequence with Myers Algorithm on Edit Graph

+

Edit Graph

+

MDA uses an Edit Graph to solve the LCS/SES problem. Below is a illustration depicting an edit graph:

+

+

The x-axis at the top of the graph represents one of the sequences, X. The y-axis at the left side of the graph represents the other sequence, Y. Hence, the two sequences in question is the following:

+
X = [A, B, C, A, B, B, A]
+Y = [C, B, A, B, A, C]
+
+

MDA generates the edit graph through the following steps:

+
    +
  1. Line the element of sequence X on the x axis. And do for Y on the y axis.
    2. +
  2. Make grid and vertex at each point in the grid (x, y), x in [0, N] and y in [0, M]. N is the length of sequence X, M is of Y
    3. +
  3. Line for x - y = k, this line called k-line. Black dot line is this and pink number is the value of k.
    4. +
  4. Check the points (i, j), where X[i] = Y[j], called match point, light green one.
    5. +
  5. Connect vertex (i - 1, j - 1) and vertex (i, j), where (i, j) is match point, then diagonal edge appears.
    6. +
+

Each elements on the figure shows that,

+ +
+

Note: Here, the sequences' start index is 1 not 0, so X[1] = A, Y[1] = C

+
+

We discuss about which path is the shortest from source to sink. Can move on the edges on the graph. I mean we can move on the grid, horizontal and vertical edges, and the diagonal edges.

+

The movements are compatible with the Edit Scripts, insert or delete. The word Edit Scripts appeared here, as referred at Introduction, SES is Shortest Edit Scripts.

+

Let's get back on track. On this edit graph, the horizontal movement to vertex (i, j) is compatible with the script delete at index i from X, the vertical movement to vertex (i, j) is compatible with the script insert the element of Y at index j to immediately after the element of X at index i. How about for the diagonal movement?. This movement to vertex (i, j) means X[i] = Y[j], so no script needs.

+ +

Next, add cost 1 for non-diagonal movement, because they can be compatible with script. And 0 for diagonal movement, same means no script.

+

The total cost for the minimum path, exploring from source to sink, is the same as the length of the Longest Common Subsequence or Shortest Edit Script.

+

So, LCS/SES problem can be solved by finding the shortest path from source to sink.

+

Myers Algorithm

+

As mentioned above, the problem of finding a shortest edit script can be reduced to finding a path from source (0, 0) to sink (N, M) with the fewest number of horizontal and vertical edges. Let D-path be a path starting at source that has exactly D non-diagonal edges, or must move non-diagonally D-times.

+

For example, A 0-path consists solely of diagonal edges. This means both sequences are completely same.

+

By a simple induction, D-path must consist of a (D-1)-path followed by a non-diagonal edge and then diagonal edges, which called snake. The minimum value of D is 0, both sequences being same. To the contrary, the maximum value of D is N + M because delete all elements from X and insert all elements from Y to X is the worst case edit scripts. For getting D, or the length of SES, running loop from 0 to N + M is enough.

+
for D in 0...N + M
+

Next, thinking about, where is the furthest reaching point for D-path on k-line. Like below, moving horizontally from k-line reaches (k+1)-line, moving vertically from k-line reaches (k-1)-line. Red chalky line shows that.

+

+

So, threre are several end points of D-path, or D-path can end on several k-line. We need the information to get the next path ((D+1)-path) as mentioned above. In fact, D-path must end on
+k-line, where k in { -D, -D + 2, ....., D - 2, D }. This is so simple, starting point, source is (0, 0) on (k=0)-line. D is the number of non-diagonal edges and non-diagonal movement changes current k-line to (kpm1)-line. Because 0 is even number, if D is even number D-path will end on (even_k)-line, if D is odd number D-path will end on (odd_k)-line.

+

Searching loop outline will be below.

+
for D in 0...N + M {
+    for k in stride(from: -D, through: D, by: 2) {
+        //Find the end point of the furthest reaching D-path in k-line.
+        if furthestReachingX == N && furthestReachingY == M {
+            // The D-path is the shortest path
+            // D is the length of Shortest Edit Script
+            return
+        }
+    }
+}
+

The D-path on k-line can be decomposed into

+ +

The Myers Algorithm key point are these.

+ +

thanks for these, the number of calculation become less.

+
public struct MyersDifferenceAlgorithm<E: Equatable> {
+    public static func calculateShortestEditDistance(from fromArray: Array<E>, to toArray: Array<E>) -> Int {
+        let fromCount = fromArray.count
+        let toCount = toArray.count
+        let totalCount = toCount + fromCount
+        var furthestReaching = Array(repeating: 0, count: 2 * totalCount + 1)
+
+        let isReachedAtSink: (Int, Int) -> Bool = { x, y in
+            return x == fromCount && y == toCount
+        }
+
+        let snake: (Int, Int, Int) -> Int = { x, D, k in
+            var _x = x
+            while _x < fromCount && _x - k < toCount && fromArray[_x] == toArray[_x - k] {
+                _x += 1
+            }
+            return _x
+        }
+
+        for D in 0...totalCount {
+            for k in stride(from: -D, through: D, by: 2) {
+                let index = k + totalCount
+            
+                // (x, D, k) => the x position on the k_line where the number of scripts is D
+                // scripts means insertion or deletion
+                var x = 0
+                if D == 0 { }
+                    // k == -D, D will be the boundary k_line
+                    // when k == -D, moving right on the Edit Graph(is delete script) from k - 1_line where D - 1 is unavailable.
+                    // when k == D, moving bottom on the Edit Graph(is insert script) from k + 1_line where D - 1 is unavailable.
+                    // furthestReaching x position has higher calculating priority. (x, D - 1, k - 1), (x, D - 1, k + 1)
+                else if k == -D || k != D && furthestReaching[index - 1] < furthestReaching[index + 1] {
+                    // Getting initial x position
+                    // ,using the furthestReaching X position on the k + 1_line where D - 1
+                    // ,meaning get (x, D, k) by (x, D - 1, k + 1) + moving bottom + snake
+                    // this moving bottom on the edit graph is compatible with insert script
+                    x = furthestReaching[index + 1]
+                } else {
+                    // Getting initial x position
+                    // ,using the futrhest X position on the k - 1_line where D - 1
+                    // ,meaning get (x, D, k) by (x, D - 1, k - 1) + moving right + snake
+                    // this moving right on the edit graph is compatible with delete script
+                    x = furthestReaching[index - 1] + 1
+                }
+                
+                // snake
+                // diagonal moving can be performed with 0 cost.
+                // `same` script is needed ?
+                let _x = snake(x, D, k)
+                
+                if isReachedAtSink(_x, _x - k) { return D }
+                furthestReaching[index] = _x
+            }
+        }
+
+        fatalError("Never comes here")
+    }
+}
+ + diff --git a/Naive Bayes Classifier/index.html b/Naive Bayes Classifier/index.html new file mode 100644 index 000000000..ddc6c8fdc --- /dev/null +++ b/Naive Bayes Classifier/index.html @@ -0,0 +1,131 @@ + + + Naive Bayes Classifier + + + +

Naive Bayes Classifier

+
+

Disclaimer: Do not get scared of complicated formulas or terms, I will describe them right after I use them. Also the math skills you need to understand this are very basic.

+
+

The goal of a classifier is to predict the class of a given data entry based on previously fed data and its features.

+

Now what is a class or a feature? The best I can do is to describe it with a table.
+This is a dataset that uses height, weight and foot size of a person to illustrate the relationship between those values and the sex.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sexheight (feet)weight(lbs)foot size (inches)
male618012
male5.9219011
male5.5817012
male5.9216510
female51006
female5.51508
female5.421307
female5.751509
+

The classes of this table is the data in the sex column (male/female). You "classify" the rest of the data and bind them to a sex.

+

The features of this table are the labels of the other columns (height, weight, foot size) and the numbers right under the labels.

+

Now that I've told you what a classifier is I will tell you what exactly a Naive Bayes classifier is. There are a lot of other classifiers out there but what's so special about this specific is that it only needs a very small dataset to get good results. The others like Random Forests normally need a very large dataset.

+

Why isn't this algorithm used more you might ask (or not). Because it is normally outperformed in accuracy by Random Forests or Boosted Trees.

+

Theory

+

The Naive Bayes classifier utilizes the Bayes Theorem (as its name suggests) which looks like this.

+

+

P always means the probability of something.

+

A is the class, B is the data depending on a feature and the pipe symbol means given.

+

P(A | B) therefore is: probability of the class given the data (which is dependent on the feature).

+

This is all you have to know about the Bayes Theorem. The important thing for us is now how to calculate all those variables, plug them into this formula and you are ready to classify data.

+

P(A)

+

This is the probability of the class. To get back to the example I gave before: Let's say we want to classify this data entry:

+ + + + + + + + + + + + + + + +
height (feet)weight(lbs)foot size (inches)
61308
+

What Naive Bayes classifier now does: it checks the probability for every class possible which is in our case either male or female. Look back at the original table and count the male and the female entries. Then divide them by the overall count of data entries.

+

P(male) = 4 / 8 = 0.5

+

P(female) = 4 / 8 = 0.5

+

This should be a very easy task to do. Basically just the probability of all classes.

+

P(B)

+

This variable is not needed in a Naive Bayes classifier. It is the probability of the data. It does not change, therefore it is a constant. And what can you do with a constant? Exactly! Discard it. This saves time and code.

+

P(B | A)

+

This is the probability of the data given the class. To calculate this I have to introduce you to the subtypes of NB. You have to decide which you use depending on your data which you want to classify.

+

Gaussian Naive Bayes

+

If you have a dataset like the one I showed you before (continuous features -> Doubles) you have to use this subtype. There are 3 formulas you need for Gaussian NB to calculate P(B | A).

+

mean

+

standard deviation

+

normal distribution

+

and P(x | y) = P(B | A)

+

Again, very complicated looking formulas but they are very easy. The first formula with µ is just the mean of the data (adding all data points and dividing them by the count). The second with σ is the standard deviation. You might have heard of it somewhen in school. It is just the sum of all values minus the mean, squared and that divided by the count of the data minus 1 and a sqaure root around it. The third equation is the Gaussian or normal distribution if you want to read more about it I suggest reading this.

+

Why the Gaussian distribution? Because we assume that the continuous values associated with each class are distributed according to the Gaussian distribution. Simple as that.

+

Multinomial Naive Bayes

+

What do we do if we have this for examples:

+

tennis or golf

+

We can't just calculate the mean of sunny, overcast and rainy. This is why we need the categorical model which is the multinomial NB. This is the last formula, I promise!

+

multinomial

+

Now this is the number of times feature i appears in a sample N of class y in the data set divided by the count of the sample just depending on the class y. That θ is also just a fancy way of writing P(B | A).

+

You might have noticed that there is still the α in this formula. This solves a problem called "zero-frequency-problem". Because what happens if there is no sample with feature i and class y? The whole equation would result in 0 (because 0 / something is always 0). This is a huge problem but there is a simple solution to this. Just add 1 to any count of the sample (α = 1).

+

Those formulas in action

+

Enough talking! This is the code. If you want a deeper explanation of how the code works just look at the Playground I provided.

+

code example

+

Written for Swift Algorithm Club by Philipp Gabriel

+ + diff --git a/Octree/index.html b/Octree/index.html new file mode 100644 index 000000000..1d9a1bff2 --- /dev/null +++ b/Octree/index.html @@ -0,0 +1,23 @@ + + + Octree + + + +

OcTree

+

An octree is a tree in which each internal (not leaf) node has eight children. Often used for collision detection in games for example.

+

Problem

+

Consider the following problem: your need to store a number of objects in 3D space (each at a certain location with X, Y and Z coordinates) and then you need to answer which objects lie in a certain 3D region. A naive solution would be to store the points inside an array and then iterate over the points and check each one individually. This solution runs in O(n) though.

+

A Better Approach

+

Octrees are most commonly used to partition a three-dimensional space by recursively subdividing it into 8 regions. Let's see how we can use an Octree to store some values.

+

Each node in the tree represents a box-like region. Leaf nodes store a single point in that region with an array of objects assigned to that point.

+

Once an object within the same region (but at a different point) is added the leaf node turns into an internal node and 8 child nodes (leaves) are added to it. All points previously contained in the node are passed to its corresponding children and stored. Thus only leaves contain actual points and values.

+

To find the points that lie in a given region we can now traverse the tree from top to bottom and collect the suitable points from nodes.

+

Both adding a point and searching can still take up to O(n) in the worst case, since the tree isn't balanced in any way. However, on average it runs significantly faster (something comparable to O(log n)).

+

See also

+

More info on Wiki
+Apple's implementation of GKOctree

+

Written for Swift Algorithm Club by Jaap Wijnen
+*Heavily inspired by Timur Galimov's Quadtree implementation and Apple's GKOctree implementation

+ + diff --git a/Ordered Array/index.html b/Ordered Array/index.html new file mode 100644 index 000000000..f682f51df --- /dev/null +++ b/Ordered Array/index.html @@ -0,0 +1,98 @@ + + + Ordered Array + + + +

Ordered Array

+

This is an array that is always sorted from low to high. Whenever you add a new item to this array, it is inserted in its sorted position.

+

An ordered array is useful for when you want your data to be sorted and you're inserting new items relatively rarely. In that case, it's faster than sorting the entire array. However, if you need to change the array often, it's probably faster to use a regular array and sort it manually.

+

The implementation is quite basic. It's simply a wrapper around Swift's built-in array:

+
public struct OrderedArray<T: Comparable> {
+  fileprivate var array = [T]()
+
+  public init(array: [T]) {
+    self.array = array.sorted()
+  }
+
+  public var isEmpty: Bool {
+    return array.isEmpty
+  }
+
+  public var count: Int {
+    return array.count
+  }
+
+  public subscript(index: Int) -> T {
+    return array[index]
+  }
+
+  public mutating func removeAtIndex(index: Int) -> T {
+    return array.remove(at: index)
+  }
+
+  public mutating func removeAll() {
+    array.removeAll()
+  }
+}
+
+extension OrderedArray: CustomStringConvertible {
+  public var description: String {
+    return array.description
+  }
+}
+

As you can see, all these methods simply call the corresponding method on the internal array variable.

+

What remains is the insert() function. Here is an initial stab at it:

+
  public mutating func insert(_ newElement: T) -> Int {
+    let i = findInsertionPoint(newElement)
+    array.insert(newElement, at: i)
+    return i
+  }
+
+  private func findInsertionPoint(_ newElement: T) -> Int {
+    for i in 0..<array.count {
+      if newElement <= array[i] {
+        return i
+      }
+    }
+    return array.count  // insert at the end
+  }
+

The helper function findInsertionPoint() simply iterates through the entire array, looking for the right place to insert the new element.

+
+

Note: Quite conveniently, array.insert(... atIndex: array.count) adds the new object to the end of the array, so if no suitable insertion point was found we can simply return array.count as the index.

+
+

Here's how you can test it in a playground:

+
var a = OrderedArray<Int>(array: [5, 1, 3, 9, 7, -1])
+a              // [-1, 1, 3, 5, 7, 9]
+
+a.insert(4)    // inserted at index 3
+a              // [-1, 1, 3, 4, 5, 7, 9]
+
+a.insert(-2)   // inserted at index 0
+a.insert(10)   // inserted at index 8
+a              // [-2, -1, 1, 3, 4, 5, 7, 9, 10]
+

The array's contents will always be sorted from low to high, now matter what.

+

Unfortunately, the current findInsertionPoint() function is a bit slow. In the worst case, it needs to scan through the entire array. We can speed this up by using a binary search to find the insertion point.

+

Here is the new version:

+
  private func findInsertionPoint(_ newElement: T) -> Int {
+    var startIndex = 0
+    var endIndex = array.count
+
+    while startIndex < endIndex {
+        let midIndex = startIndex + (endIndex - startIndex) / 2
+        if array[midIndex] == newElement {
+            return midIndex
+        } else if array[midIndex] < newElement {
+            startIndex = midIndex + 1
+        } else {
+            endIndex = midIndex
+        }
+    }
+    return startIndex
+  }
+

The big difference with a regular binary search is that this doesn't return nil when the value can't be found, but the array index where the element would have been. That's where we insert the new object.

+

Note that using binary search doesn't change the worst-case running time complexity of insert(). The binary search itself takes only O(log n) time, but inserting a new object in the middle of an array still involves shifting all remaining elements in memory. So overall, the time complexity is still O(n). But in practice this new version definitely is a lot faster, especially on large arrays.

+

A more complete and production ready SortedArray is avalible from Ole Begemann. The accompanying article explains the advantages and tradeoffs.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Ordered Set/index.html b/Ordered Set/index.html new file mode 100644 index 000000000..4708ca8ef --- /dev/null +++ b/Ordered Set/index.html @@ -0,0 +1,96 @@ + + + Ordered Set + + + +

Ordered Set

+

Let's look into how to implement Ordered Set.

+

Here is the example about how it works

+
let s = AppleOrderedSet<Int>()
+
+s.add(1)
+s.add(2)
+s.add(-1)
+s.add(0)
+s.insert(4, at: 3)
+
+print(s.all()) // [1, 2, -1, 4, 0]
+
+s.set(-1, at: 0) // We already have -1 in index: 2, so we will do nothing here
+
+print(s.all()) // [1, 2, -1, 4, 0]
+
+s.remove(-1)
+
+print(s.all()) // [1, 2, 4, 0]
+
+print(s.object(at: 1)) // 2
+
+print(s.object(at: 2)) // 4
+

The significant difference is the the array is not sorted. The elements in the array are the same when insert them. Image the array without duplicates and with O(logn) or O(1) search time.

+

The idea here is using a data structure to provide O(1) or O(logn) time complexity, so it's easy to think about hash table.

+
var indexOfKey: [T: Int]
+var objects: [T]
+

indexOfKey is used to track the index of the element. objects is array holding elements.

+

We will go through some key functions details here.

+

Add

+

Update indexOfKey and insert element in the end of objects

+
// O(1)
+public func add(_ object: T) {
+	guard indexOfKey[object] == nil else {
+		return
+	}
+
+	objects.append(object)
+	indexOfKey[object] = objects.count - 1
+}
+

Insert

+

Insert in a random place of the array will cost O(n) time.

+
// O(n)
+public func insert(_ object: T, at index: Int) {
+	assert(index < objects.count, "Index should be smaller than object count")
+	assert(index >= 0, "Index should be bigger than 0")
+
+	guard indexOfKey[object] == nil else {
+		return
+	}
+
+	objects.insert(object, at: index)
+	indexOfKey[object] = index
+	for i in index+1..<objects.count {
+		indexOfKey[objects[i]] = i
+	}
+}
+

Set

+

If the object already existed in the OrderedSet, do nothing. Otherwise, we need to update the indexOfkey and objects.

+
// O(1)
+public func set(_ object: T, at index: Int) {
+	assert(index < objects.count, "Index should be smaller than object count")
+	assert(index >= 0, "Index should be bigger than 0")
+
+	guard indexOfKey[object] == nil else {
+		return
+	}
+
+	indexOfKey.removeValue(forKey: objects[index])
+	indexOfKey[object] = index
+	objects[index] = object
+}
+

Remove

+

Remove element in the array will cost O(n). At the same time, we need to update all elements's index after the removed element.

+
// O(n)
+public func remove(_ object: T) {
+	guard let index = indexOfKey[object] else {
+		return 
+	}
+
+	indexOfKey.removeValue(forKey: object)
+	objects.remove(at: index)
+	for i in index..<objects.count {
+		indexOfKey[objects[i]] = i
+	}
+}
+

Written By Kai Chen

+ + diff --git a/Palindromes/index.html b/Palindromes/index.html new file mode 100644 index 000000000..981a189b4 --- /dev/null +++ b/Palindromes/index.html @@ -0,0 +1,86 @@ + + + Palindromes + + + +

Palindromes

+

A palindrome is a word or phrase that is spelled the exact same when reading it forwards or backward. Palindromes are allowed to be lowercase or uppercase, contain spaces, punctuation, and word dividers.

+

Algorithms that check for palindromes are a common programming interview question.

+

Example

+

The word racecar is a valid palindrome, as it is a word spelled the same when backgrounds and forwards. The examples below shows valid cases of the palindrome racecar.

+
raceCar
+r a c e c a r
+r?a?c?e?c?a?r?
+RACEcar
+
+

Algorithm

+

To check for palindromes, a string's characters are compared starting from the beginning and end then moving inward toward the middle of the string while maintaining the same distance apart. In this implementation of a palindrome algorithm, recursion is used to check each of the characters on the left-hand side and right-hand side moving inward.

+

The code

+

Here is a recursive implementation of this in Swift:

+
func isPalindrome(_ str: String) -> Bool {
+  let strippedString = str.replacingOccurrences(of: "\W", with: "", options: .regularExpression, range: nil)
+  let length = strippedString.count
+
+  if length > 1 {
+    return palindrome(strippedString.lowercased(), left: 0, right: length - 1)
+  }
+
+  return false
+}
+
+private func palindrome(_ str: String, left: Int, right: Int) -> Bool {
+  if left >= right {
+    return true
+  }
+
+  let lhs = str[str.index(str.startIndex, offsetBy: left)]
+  let rhs = str[str.index(str.startIndex, offsetBy: right)]
+
+  if lhs != rhs {
+    return false
+  }
+
+  return palindrome(str, left: left + 1, right: right - 1)
+}
+

This algorithm has a two-step process.

+
    +
  1. The first step is to pass the string to validate as a palindrome into the isPalindrome method. This method first removes occurrences of non-word pattern matches \W Regex reference. It is written with two \ to escape the \ in the String literal.
    2. +
+
let strippedString = str.replacingOccurrences(of: "\W", with: "", options: .regularExpression, range: nil)
+

The length of the string is then checked to make sure that the string after being stripped of non-word characters is still in a valid length. It is then passed into the next step after being lowercased.

+
    +
  1. The second step is to pass the string in a recursive method. This method takes a string, a left index, and a right index. The method checks the characters of the string using the indexes to compare each character on both sides. The method checks if the left is greater or equal to the right if so the entire string has been run through without returning false so the string is equal on both sides thus returning true.
    2. +
+
if left >= right {
+  return true
+}
+

If the check doesn't pass it continues to get the characters at the specified indexes and compare each. If they are not the same the method returns false and exits.

+
let lhs = str[str.index(str.startIndex, offsetBy: left)]
+let rhs = str[str.index(str.startIndex, offsetBy: right)]
+
+if lhs != rhs {
+  return false
+}
+

If they are the same the method calls itself again and updates the indexes accordingly to continue to check the rest of the string.

+
return palindrome(str, left: left + 1, right: right - 1)
+

Step 1:
+race?C ar -> raceCar -> racecar

+

Step 2:

+
|     |
+racecar -> r == r
+
+ |   |
+racecar -> a == a
+
+  | |
+racecar -> c == c
+
+   |
+racecar -> left index == right index -> return true
+
+

Additional Resources

+

Palindrome Wikipedia

+

Written by Joshua Alvarado

+ + diff --git a/Points Lines Planes/index.html b/Points Lines Planes/index.html new file mode 100644 index 000000000..d92c7988f --- /dev/null +++ b/Points Lines Planes/index.html @@ -0,0 +1,34 @@ + + + Points Lines Planes + + + +

Points Lines (Planes)

+

This implements data structures for points lines and planes(not yet) in (for now) 2D space and a few functions to play around with them. This was originally written to improve on the Convex Hull algorithm but I thought it might be a nice addition in itself. Im planning to add 3D implementations as well.

+

implementation

+

Two structs are implemented the Point2D and Line2D.

+
struct Point2D: {
+  var x: Double
+  var y: double
+}
+
+
struct Line2D {
+  var slope: Slope
+  var offset: Double
+  var direction: Direction
+}
+
+

Here Slope is an enum to account for vertical lines.
+slope is infinite for vertical lines, offset is the x coordinate where the line crosses the line y=0.
+slope is finite for any other line and contains a double with the actual value of the slope.

+
enum Slope {
+  case finite(slope: Double)
+  case infinite(offset: Double)
+}
+
+

Line2D also contains a Direction enum. This is introduced in order to make lines directional in order to be able to determine what is left and right of a certain line. It is .increasing if the line points in positive y direction and .decreasing if it points in the negative y direction.

+

Line2D's offset is the the y-coordinate where the line crosses the vertical x=0 line.

+

Written for the Swift Algorithm Club by Jaap Wijnen.

+ + diff --git a/Priority Queue/index.html b/Priority Queue/index.html new file mode 100644 index 000000000..fc6affc5a --- /dev/null +++ b/Priority Queue/index.html @@ -0,0 +1,73 @@ + + + Priority Queue + + + +

Priority Queue

+

A priority queue is a queue where the most important element is always at the front.

+

The queue can be a max-priority queue (largest element first) or a min-priority queue (smallest element first).

+

Why use a priority queue?

+

Priority queues are useful for algorithms that need to process a (large) number of items and where you repeatedly need to identify which one is now the biggest or smallest -- or however you define "most important".

+

Examples of algorithms that can benefit from a priority queue:

+ +

With a regular queue or plain old array you'd need to scan the entire sequence over and over to find the next largest item. A priority queue is optimized for this sort of thing.

+

What can you do with a priority queue?

+

Common operations on a priority queue:

+ +

How to implement a priority queue

+

There are different ways to implement priority queues:

+ +

Here's a Swift priority queue based on a heap:

+
public struct PriorityQueue<T> {
+  fileprivate var heap: Heap<T>
+
+  public init(sort: (T, T) -> Bool) {
+    heap = Heap(sort: sort)
+  }
+
+  public var isEmpty: Bool {
+    return heap.isEmpty
+  }
+
+  public var count: Int {
+    return heap.count
+  }
+
+  public func peek() -> T? {
+    return heap.peek()
+  }
+
+  public mutating func enqueue(element: T) {
+    heap.insert(element)
+  }
+
+  public mutating func dequeue() -> T? {
+    return heap.remove()
+  }
+
+  public mutating func changePriority(index i: Int, value: T) {
+    return heap.replace(index: i, value: value)
+  }
+}
+

As you can see, there's nothing much to it. Making a priority queue is easy if you have a heap because a heap is pretty much a priority queue.

+

See also

+

Priority Queue on Wikipedia

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/QuadTree/index.html b/QuadTree/index.html new file mode 100644 index 000000000..eb897a055 --- /dev/null +++ b/QuadTree/index.html @@ -0,0 +1,144 @@ + + + QuadTree + + + +

QuadTree

+

A quadtree is a tree in which each internal (not leaf) node has four children.

+

+

Problem

+

Consider the following problem: your need to store a number of points (each point is a pair of X and Y coordinates) and then you need to answer which points lie in a certain rectangular region. A naive solution would be to store the points inside an array and then iterate over the points and check each one individually. This solution runs in O(n) though.

+

A Better Approach

+

Quadtrees are most commonly used to partition a two-dimensional space by recursively subdividing it into four regions(quadrants). Let's see how we can use a Quadtree to store the points.

+

Each node in the tree represents a rectangular region and stores a limited number(maxPointCapacity) of points that all lie in its region.

+
class QuadTreeNode {
+
+  enum NodeType {
+    case leaf
+    case `internal`(children: Children)
+  }
+
+  struct Children {
+    let leftTop: QuadTreeNode
+    let leftBottom: QuadTreeNode
+    let rightTop: QuadTreeNode
+    let rightBottom: QuadTreeNode
+
+    ...
+  }
+
+  var points: [Point] = []
+  let rect: Rect
+  var type: NodeType = .leaf
+
+  static let maxPointCapacity = 3
+
+  init(rect: Rect) {
+    self.rect = rect
+  }
+
+  ...
+}
+
+

Once the limit in a leaf node is reached, four child nodes are added to the node and they represent topLeft, topRight, bottomLeft, bottomRight quadrants of the node's rect; each of the consequent points in the rect will be passed to one of the children. Thus, new points are always added to leaf nodes.

+
extension QuadTreeNode {
+
+  @discardableResult
+  func add(point: Point) -> Bool {
+
+    if !rect.contains(point: point) {
+      return false
+    }
+
+    switch type {
+    case .internal(let children):
+      // pass the point to one of the children
+      for child in children {
+        if child.add(point: point) {
+          return true
+        }
+      }
+      return false // should never happen
+    case .leaf:
+      points.append(point)
+      // if the max capacity was reached, become an internal node
+      if points.count == QuadTreeNode.maxPointCapacity {
+        subdivide()
+      }
+    }
+    return true
+  }
+
+  private func subdivide() {
+    switch type {
+    case .leaf:
+      type = .internal(children: Children(parentNode: self))
+    case .internal:
+      preconditionFailure("Calling subdivide on an internal node")
+    }
+  }
+}
+
+extension Children {
+
+  init(parentNode: QuadTreeNode) {
+    leftTop = QuadTreeNode(rect: parentNode.rect.leftTopRect)
+    leftBottom = QuadTreeNode(rect: parentNode.rect.leftBottomRect)
+    rightTop = QuadTreeNode(rect: parentNode.rect.rightTopRect)
+    rightBottom = QuadTreeNode(rect: parentNode.rect.rightBottomRect)
+  }
+}
+
+

To find the points that lie in a given region we can now traverse the tree from top to bottom and collect the suitable points from nodes.

+
class QuadTree {
+
+  ...
+
+  let root: QuadTreeNode
+
+   public func points(inRect rect: Rect) -> [Point] {
+    return root.points(inRect: rect)
+  }
+}
+
+extension QuadTreeNode {
+  func points(inRect rect: Rect) -> [Point] {
+
+    // if the node's rect and the given rect don't intersect, return an empty array,
+    // because there can't be any points that lie the node's (or its children's) rect and
+    // in the given rect
+    if !self.rect.intersects(rect: rect) {
+      return []
+    }
+
+    var result: [Point] = []
+
+    // collect the node's points that lie in the rect
+    for point in points {
+      if rect.contains(point: point) {
+        result.append(point)
+      }
+    }
+
+    switch type {
+    case .leaf:
+      break
+    case .internal(children: let children):
+      // recursively add children's points that lie in the rect
+      for childNode in children {
+        result.append(contentsOf: childNode.points(inRect: rect))
+      }
+    }
+
+    return result
+  }
+}
+
+

Both adding a point and searching can still take up to O(n) in the worst case, since the tree isn't balanced in any way. However, on average it runs significantly faster (something comparable to O(log n)).

+

See also

+

Displaying a large amount of objects in a MapView - a great use case for a Quadtree (Thoughtbot Article)

+

More info on Wikipedia

+

Written for Swift Algorithm Club by Timur Galimov

+ + diff --git a/Queue/index.html b/Queue/index.html new file mode 100644 index 000000000..b0c4c1c86 --- /dev/null +++ b/Queue/index.html @@ -0,0 +1,191 @@ + + + Queue + + + +

Queue

+
+

This topic has been tutorialized here

+
+

A queue is a list where you can only insert new items at the back and remove items from the front. This ensures that the first item you enqueue is also the first item you dequeue. First come, first serve!

+

Why would you need this? Well, in many algorithms you want to add objects to a temporary list and pull them off this list later. Often the order in which you add and remove these objects matters.

+

A queue gives you a FIFO or first-in, first-out order. The element you inserted first is the first one to come out. It is only fair! (A similar data structure, the stack, is LIFO or last-in first-out.)

+

Here is an example to enqueue a number:

+
queue.enqueue(10)
+

The queue is now [ 10 ]. Add the next number to the queue:

+
queue.enqueue(3)
+

The queue is now [ 10, 3 ]. Add one more number:

+
queue.enqueue(57)
+

The queue is now [ 10, 3, 57 ]. Let's dequeue to pull the first element off the front of the queue:

+
queue.dequeue()
+

This returns 10 because that was the first number we inserted. The queue is now [ 3, 57 ]. Everyone moved up by one place.

+
queue.dequeue()
+

This returns 3, the next dequeue returns 57, and so on. If the queue is empty, dequeuing returns nil or in some implementations it gives an error message.

+
+

Note: A queue is not always the best choice. If the order in which the items are added and removed from the list is not important, you can use a stack instead of a queue. Stacks are simpler and faster.

+
+

The code

+

Here is a simplistic implementation of a queue in Swift. It is a wrapper around an array to enqueue, dequeue, and peek at the front-most item:

+
public struct Queue<T> {
+  fileprivate var array = [T]()
+
+  public var isEmpty: Bool {
+    return array.isEmpty
+  }
+  
+  public var count: Int {
+    return array.count
+  }
+
+  public mutating func enqueue(_ element: T) {
+    array.append(element)
+  }
+  
+  public mutating func dequeue() -> T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array.removeFirst()
+    }
+  }
+  
+  public var front: T? {
+    return array.first
+  }
+}
+

This queue works well, but it is not optimal.

+

Enqueuing is an O(1) operation because adding to the end of an array always takes the same amount of time regardless of the size of the array.

+

You might be wondering why appending items to an array is O(1) or a constant-time operation. That is because an array in Swift always has some empty space at the end. If we do the following:

+
var queue = Queue<String>()
+queue.enqueue("Ada")
+queue.enqueue("Steve")
+queue.enqueue("Tim")
+

then the array might actually look like this:

+
[ "Ada", "Steve", "Tim", xxx, xxx, xxx ]
+
+

where xxx is memory that is reserved but not filled in yet. Adding a new element to the array overwrites the next unused spot:

+
[ "Ada", "Steve", "Tim", "Grace", xxx, xxx ]
+
+

This results by copying memory from one place to another which is a constant-time operation.

+

There are only a limited number of unused spots at the end of the array. When the last xxx gets used, and you want to add another item, the array needs to resize to make more room.

+

Resizing includes allocating new memory and copying all the existing data over to the new array. This is an O(n) process which is relatively slow. Since it happens occasionally, the time for appending a new element to the end of the array is still O(1) on average or O(1) "amortized".

+

The story for dequeueing is different. To dequeue, we remove the element from the beginning of the array. This is always an O(n) operation because it requires all remaining array elements to be shifted in memory.

+

In our example, dequeuing the first element "Ada" copies "Steve" in the place of "Ada", "Tim" in the place of "Steve", and "Grace" in the place of "Tim":

+
before   [ "Ada", "Steve", "Tim", "Grace", xxx, xxx ]
+                   /       /      /
+                  /       /      /
+                 /       /      /
+                /       /      /
+ after   [ "Steve", "Tim", "Grace", xxx, xxx, xxx ]
+
+

Moving all these elements in memory is always an O(n) operation. So with our simple implementation of a queue, enqueuing is efficient, but dequeueing leaves something to be desired...

+

A more efficient queue

+

To make dequeuing efficient, we can also reserve some extra free space but this time at the front of the array. We must write this code ourselves because the built-in Swift array does not support it.

+

The main idea is whenever we dequeue an item, we do not shift the contents of the array to the front (slow) but mark the item's position in the array as empty (fast). After dequeuing "Ada", the array is:

+
[ xxx, "Steve", "Tim", "Grace", xxx, xxx ]
+
+

After dequeuing "Steve", the array is:

+
[ xxx, xxx, "Tim", "Grace", xxx, xxx ]
+
+

Because these empty spots at the front never get reused, you can periodically trim the array by moving the remaining elements to the front:

+
[ "Tim", "Grace", xxx, xxx, xxx, xxx ]
+
+

This trimming procedure involves shifting memory which is an O(n) operation. Because this only happens once in a while, dequeuing is O(1) on average.

+

Here is how you can implement this version of Queue:

+
public struct Queue<T> {
+  fileprivate var array = [T?]()
+  fileprivate var head = 0
+  
+  public var isEmpty: Bool {
+    return count == 0
+  }
+
+  public var count: Int {
+    return array.count - head
+  }
+  
+  public mutating func enqueue(_ element: T) {
+    array.append(element)
+  }
+  
+  public mutating func dequeue() -> T? {
+    guard head < array.count, let element = array[head] else { return nil }
+
+    array[head] = nil
+    head += 1
+
+    let percentage = Double(head)/Double(array.count)
+    if array.count > 50 && percentage > 0.25 {
+      array.removeFirst(head)
+      head = 0
+    }
+    
+    return element
+  }
+  
+  public var front: T? {
+    if isEmpty {
+      return nil
+    } else {
+      return array[head]
+    }
+  }
+}
+

The array now stores objects of type T? instead of just T because we need to mark array elements as being empty. The head variable is the index in the array of the front-most object.

+

Most of the new functionality sits in dequeue(). When we dequeue an item, we first set array[head] to nil to remove the object from the array. Then, we increment head because the next item has become the front one.

+

We go from this:

+
[ "Ada", "Steve", "Tim", "Grace", xxx, xxx ]
+  head
+
+

to this:

+
[ xxx, "Steve", "Tim", "Grace", xxx, xxx ]
+        head
+
+

It is like if in a supermarket the people in the checkout lane do not shuffle forward towards the cash register, but the cash register moves up the queue.

+

If we never remove those empty spots at the front then the array will keep growing as we enqueue and dequeue elements. To periodically trim down the array, we do the following:

+
    let percentage = Double(head)/Double(array.count)
+    if array.count > 50 && percentage > 0.25 {
+      array.removeFirst(head)
+      head = 0
+    }
+

This calculates the percentage of empty spots at the beginning as a ratio of the total array size. If more than 25% of the array is unused, we chop off that wasted space. However, if the array is small we do not resize it all the time, so there must be at least 50 elements in the array before we try to trim it.

+
+

Note: I just pulled these numbers out of thin air -- you may need to tweak them based on the behavior of your app in a production environment.

+
+

To test this in a playground, do the following:

+
var q = Queue<String>()
+q.array                   // [] empty array
+
+q.enqueue("Ada")
+q.enqueue("Steve")
+q.enqueue("Tim")
+q.array             // [{Some "Ada"}, {Some "Steve"}, {Some "Tim"}]
+q.count             // 3
+
+q.dequeue()         // "Ada"
+q.array             // [nil, {Some "Steve"}, {Some "Tim"}]
+q.count             // 2
+
+q.dequeue()         // "Steve"
+q.array             // [nil, nil, {Some "Tim"}]
+q.count             // 1
+
+q.enqueue("Grace")
+q.array             // [nil, nil, {Some "Tim"}, {Some "Grace"}]
+q.count             // 2
+

To test the trimming behavior, replace the line,

+
    if array.count > 50 && percentage > 0.25 {
+

with:

+
    if head > 2 {
+

Now if you dequeue another object, the array will look as follows:

+
q.dequeue()         // "Tim"
+q.array             // [{Some "Grace"}]
+q.count             // 1
+

The nil objects at the front have been removed, and the array is no longer wasting space. This new version of Queue is not more complicated than the first one but dequeuing is now also an O(1) operation, just because we were aware about how we used the array.

+

See also

+

There are many ways to create a queue. Alternative implementations use a linked list, a circular buffer, or a heap.

+

Variations on this theme are deque, a double-ended queue where you can enqueue and dequeue at both ends, and priority queue, a sorted queue where the "most important" item is always at the front.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Quicksort/index.html b/Quicksort/index.html new file mode 100644 index 000000000..dcaf43296 --- /dev/null +++ b/Quicksort/index.html @@ -0,0 +1,354 @@ + + + Quicksort + + + +

Quicksort

+

Goal: Sort an array from low to high (or high to low).

+

Quicksort is one of the most famous algorithms in history. It was invented way back in 1959 by Tony Hoare, at a time when recursion was still a fairly nebulous concept.

+

Here's an implementation in Swift that should be easy to understand:

+
func quicksort<T: Comparable>(_ a: [T]) -> [T] {
+  guard a.count > 1 else { return a }
+
+  let pivot = a[a.count/2]
+  let less = a.filter { $0 < pivot }
+  let equal = a.filter { $0 == pivot }
+  let greater = a.filter { $0 > pivot }
+
+  return quicksort(less) + equal + quicksort(greater)
+}
+

Put this code in a playground and test it like so:

+
let list = [ 10, 0, 3, 9, 2, 14, 8, 27, 1, 5, 8, -1, 26 ]
+quicksort(list)
+

Here's how it works. When given an array, quicksort() splits it up into three parts based on a "pivot" variable. Here, the pivot is taken to be the element in the middle of the array (later on you'll see other ways to choose the pivot).

+

All the elements less than the pivot go into a new array called less. All the elements equal to the pivot go into the equal array. And you guessed it, all elements greater than the pivot go into the third array, greater. This is why the generic type T must be Comparable, so we can compare the elements with <, ==, and >.

+

Once we have these three arrays, quicksort() recursively sorts the less array and the greater array, then glues those sorted subarrays back together with the equal array to get the final result.

+

An example

+

Let's walk through the example. The array is initially:

+
[ 10, 0, 3, 9, 2, 14, 8, 27, 1, 5, 8, -1, 26 ]
+
+

First, we pick the pivot element. That is 8 because it's in the middle of the array. Now we split the array into the less, equal, and greater parts:

+
less:    [ 0, 3, 2, 1, 5, -1 ]
+equal:   [ 8, 8 ]
+greater: [ 10, 9, 14, 27, 26 ]
+
+

This is a good split because less and greater roughly contain the same number of elements. So we've picked a good pivot that chopped the array right down the middle.

+

Note that the less and greater arrays aren't sorted yet, so we call quicksort() again to sort those two subarrays. That does the exact same thing: pick a pivot and split the subarray into three even smaller parts.

+

Let's just take a look at the less array:

+
[ 0, 3, 2, 1, 5, -1 ]
+
+

The pivot element is the one in the middle, 1. (You could also have picked 2, it doesn't matter.) Again, we create three subarrays around the pivot:

+
less:    [ 0, -1 ]
+equal:   [ 1 ]
+greater: [ 3, 2, 5 ]
+
+

We're not done yet and quicksort() again is called recursively on the less and greater arrays. Let's look at less again:

+
[ 0, -1 ]
+
+

As pivot we pick -1. Now the subarrays are:

+
less:    [ ]
+equal:   [ -1 ]
+greater: [ 0 ]
+
+

The less array is empty because there was no value smaller than -1; the other arrays contain a single element each. That means we're done at this level of the recursion, and we go back up to sort the previous greater array.

+

That greater array was:

+
[ 3, 2, 5 ]
+
+

This works just the same way as before: we pick the middle element 2 as the pivot and fill up the subarrays:

+
less:    [ ]
+equal:   [ 2 ]
+greater: [ 3, 5 ]
+
+

Note that here it would have been better to pick 3 as the pivot -- we would have been done sooner. But now we have to recurse into the greater array again to make sure it is sorted. This is why picking a good pivot is important. When you pick too many "bad" pivots, quicksort actually becomes really slow. More on that below.

+

When we partition the greater subarray, we find:

+
less:    [ 3 ]
+equal:   [ 5 ]
+greater: [ ]
+
+

And now we're done at this level of the recursion because we can't split up the arrays any further.

+

This process repeats until all the subarrays have been sorted. In a picture:

+

Example

+

Now if you read the colored boxes from left to right, you get the sorted array:

+
[ -1, 0, 1, 2, 3, 5, 8, 8, 9, 10, 14, 26, 27 ]
+
+

This shows that 8 was a good initial pivot because it appears in the middle of the sorted array too.

+

I hope this makes the basic principle clear of how quicksort works. Unfortunately, this version of quicksort isn't very quick, because we filter() the same array three times. There are more clever ways to split up the array.

+

Partitioning

+

Dividing the array around the pivot is called partitioning and there are a few different partitioning schemes.

+

If the array is,

+
[ 10, 0, 3, 9, 2, 14, 8, 27, 1, 5, 8, -1, 26 ]
+
+

and we choose the middle element 8 as a pivot then after partitioning the array will look like this:

+
[ 0, 3, 2, 1, 5, -1, 8, 8, 10, 9, 14, 27, 26 ]
+  -----------------        -----------------
+  all elements < 8         all elements > 8
+
+

The key thing to realize is that after partitioning the pivot element is in its final sorted place already. The rest of the numbers are not sorted yet, they are simply partitioned around the pivot value. Quicksort partitions the array many times over, until all the values are in their final places.

+

There is no guarantee that partitioning keeps the elements in the same relative order, so after partitioning around pivot 8 you could also end up with something like this:

+
[ 3, 0, 5, 2, -1, 1, 8, 8, 14, 26, 10, 27, 9 ]
+
+

The only guarantee is that to the left of the pivot are all the smaller elements and to the right are all the larger elements. Because partitioning can change the original order of equal elements, quicksort does not produce a "stable" sort (unlike merge sort, for example). Most of the time that's not a big deal.

+

Lomuto's partitioning scheme

+

In the first example of quicksort I showed you, partitioning was done by calling Swift's filter() function three times. That is not very efficient. So let's look at a smarter partitioning algorithm that works in place, i.e. by modifying the original array.

+

Here's an implementation of Lomuto's partitioning scheme in Swift:

+
func partitionLomuto<T: Comparable>(_ a: inout [T], low: Int, high: Int) -> Int {
+  let pivot = a[high]
+
+  var i = low
+  for j in low..<high {
+    if a[j] <= pivot {
+      (a[i], a[j]) = (a[j], a[i])
+      i += 1
+    }
+  }
+
+  (a[i], a[high]) = (a[high], a[i])
+  return i
+}
+

To test this in a playground, do:

+
var list = [ 10, 0, 3, 9, 2, 14, 26, 27, 1, 5, 8, -1, 8 ]
+let p = partitionLomuto(&list, low: 0, high: list.count - 1)
+list  // show the results
+

Note that list needs to be a var because partitionLomuto() directly changes the contents of the array (it is passed as an inout parameter). That is much more efficient than allocating a new array object.

+

The low and high parameters are necessary because when this is used inside quicksort, you don't always want to (re)partition the entire array, only a limited range that becomes smaller and smaller.

+

Previously we used the middle array element as the pivot but it's important to realize that the Lomuto algorithm always uses the last element, a[high], for the pivot. Because we've been pivoting around 8 all this time, I swapped the positions of 8 and 26 in the example so that 8 is at the end of the array and is used as the pivot value here too.

+

After partitioning, the array looks like this:

+
[ 0, 3, 2, 1, 5, 8, -1, 8, 9, 10, 14, 26, 27 ]
+                        *
+
+

The variable p contains the return value of the call to partitionLomuto() and is 7. This is the index of the pivot element in the new array (marked with a star).

+

The left partition goes from 0 to p-1 and is [ 0, 3, 2, 1, 5, 8, -1 ]. The right partition goes from p+1 to the end, and is [ 9, 10, 14, 26, 27 ] (the fact that the right partition is already sorted is a coincidence).

+

You may notice something interesting... The value 8 occurs more than once in the array. One of those 8s did not end up neatly in the middle but somewhere in the left partition. That's a small downside of the Lomuto algorithm as it makes quicksort slower if there are a lot of duplicate elements.

+

So how does the Lomuto algorithm actually work? The magic happens in the for loop. This loop divides the array into four regions:

+
    +
  1. a[low...i] contains all values <= pivot
    2. +
  2. a[i+1...j-1] contains all values > pivot
    3. +
  3. a[j...high-1] are values we haven't looked at yet
    4. +
  4. a[high] is the pivot value
    5. +
+

In ASCII art the array is divided up like this:

+
[ values <= pivot | values > pivot | not looked at yet | pivot ]
+  low           i   i+1        j-1   j          high-1   high
+
+

The loop looks at each element from low to high-1 in turn. If the value of the current element is less than or equal to the pivot, it is moved into the first region using a swap.

+
+

Note: In Swift, the notation (x, y) = (y, x) is a convenient way to perform a swap between the values of x and y. You can also write swap(&x, &y).

+
+

After the loop is over, the pivot is still the last element in the array. So we swap it with the first element that is greater than the pivot. Now the pivot sits between the <= and > regions and the array is properly partitioned.

+

Let's step through the example. The array we're starting with is:

+
[| 10, 0, 3, 9, 2, 14, 26, 27, 1, 5, 8, -1 | 8 ]
+   low                                       high
+   i
+   j
+
+

Initially, the "not looked at" region stretches from index 0 to 11. The pivot is at index 12. The "values <= pivot" and "values > pivot" regions are empty, because we haven't looked at any values yet.

+

Look at the first value, 10. Is this smaller than the pivot? No, skip to the next element.

+
[| 10 | 0, 3, 9, 2, 14, 26, 27, 1, 5, 8, -1 | 8 ]
+   low                                        high
+   i
+       j
+
+

Now the "not looked at" region goes from index 1 to 11, the "values > pivot" region contains the number 10, and "values <= pivot" is still empty.

+

Look at the second value, 0. Is this smaller than the pivot? Yes, so swap 10 with 0 and move i ahead by one.

+
[ 0 | 10 | 3, 9, 2, 14, 26, 27, 1, 5, 8, -1 | 8 ]
+  low                                         high
+      i
+           j
+
+

Now "not looked at" goes from index 2 to 11, "values > pivot" still contains 10, and "values <= pivot" contains the number 0.

+

Look at the third value, 3. This is smaller than the pivot, so swap it with 10 to get:

+
[ 0, 3 | 10 | 9, 2, 14, 26, 27, 1, 5, 8, -1 | 8 ]
+  low                                         high
+         i
+             j
+
+

The "values <= pivot" region is now [ 0, 3 ]. Let's do one more... 9 is greater than the pivot, so simply skip ahead:

+
[ 0, 3 | 10, 9 | 2, 14, 26, 27, 1, 5, 8, -1 | 8 ]
+  low                                         high
+         i
+                 j
+
+

Now the "values > pivot" region contains [ 10, 9 ]. If we keep going this way, then eventually we end up with:

+
[ 0, 3, 2, 1, 5, 8, -1 | 27, 9, 10, 14, 26 | 8 ]
+  low                                        high
+                         i                   j
+
+

The final thing to do is to put the pivot into place by swapping a[i] with a[high]:

+
[ 0, 3, 2, 1, 5, 8, -1 | 8 | 9, 10, 14, 26, 27 ]
+  low                                       high
+                         i                  j
+
+

And we return i, the index of the pivot element.

+
+

Note: If you're still not entirely clear on how the algorithm works, I suggest you play with this in the playground to see exactly how the loop creates these four regions.

+
+

Let's use this partitioning scheme to build quicksort. Here's the code:

+
func quicksortLomuto<T: Comparable>(_ a: inout [T], low: Int, high: Int) {
+  if low < high {
+    let p = partitionLomuto(&a, low: low, high: high)
+    quicksortLomuto(&a, low: low, high: p - 1)
+    quicksortLomuto(&a, low: p + 1, high: high)
+  }
+}
+

This is now super simple. We first call partitionLomuto() to reorder the array around the pivot (which is always the last element from the array). And then we call quicksortLomuto() recursively to sort the left and right partitions.

+

Try it out:

+
var list = [ 10, 0, 3, 9, 2, 14, 26, 27, 1, 5, 8, -1, 8 ]
+quicksortLomuto(&list, low: 0, high: list.count - 1)
+

Lomuto's isn't the only partitioning scheme but it's probably the easiest to understand. It's not as efficient as Hoare's scheme, which requires fewer swap operations.

+

Hoare's partitioning scheme

+

This partitioning scheme is by Hoare, the inventor of quicksort.

+

Here is the code:

+
func partitionHoare<T: Comparable>(_ a: inout [T], low: Int, high: Int) -> Int {
+  let pivot = a[low]
+  var i = low - 1
+  var j = high + 1
+
+  while true {
+    repeat { j -= 1 } while a[j] > pivot
+    repeat { i += 1 } while a[i] < pivot
+
+    if i < j {
+      a.swapAt(i, j)
+    } else {
+      return j
+    }
+  }
+}
+

To test this in a playground, do:

+
var list = [ 8, 0, 3, 9, 2, 14, 10, 27, 1, 5, 8, -1, 26 ]
+let p = partitionHoare(&list, low: 0, high: list.count - 1)
+list  // show the results
+

Note that with Hoare's scheme, the pivot is always expected to be the first element in the array, a[low]. Again, we're using 8 as the pivot element.

+

The result is:

+
[ -1, 0, 3, 8, 2, 5, 1, 27, 10, 14, 9, 8, 26 ]
+
+

Note that this time the pivot isn't in the middle at all. Unlike with Lomuto's scheme, the return value is not necessarily the index of the pivot element in the new array.

+

Instead, the array is partitioned into the regions [low...p] and [p+1...high]. Here, the return value p is 6, so the two partitions are [ -1, 0, 3, 8, 2, 5, 1 ] and [ 27, 10, 14, 9, 8, 26 ].

+

The pivot is placed somewhere inside one of the two partitions, but the algorithm doesn't tell you which one or where. If the pivot value occurs more than once, then some instances may appear in the left partition and others may appear in the right partition.

+

Because of these differences, the implementation of Hoare's quicksort is slightly different:

+
func quicksortHoare<T: Comparable>(_ a: inout [T], low: Int, high: Int) {
+  if low < high {
+    let p = partitionHoare(&a, low: low, high: high)
+    quicksortHoare(&a, low: low, high: p)
+    quicksortHoare(&a, low: p + 1, high: high)
+  }
+}
+

I'll leave it as an exercise for the reader to figure out exactly how Hoare's partitioning scheme works. :-)

+

Picking a good pivot

+

Lomuto's partitioning scheme always chooses the last array element for the pivot. Hoare's scheme uses the first element. But there is no guarantee that these pivots are any good.

+

Here is what happens when you pick a bad value for the pivot. Let's say the array is,

+
[ 7, 6, 5, 4, 3, 2, 1 ]
+
+

and we're using Lomuto's scheme. The pivot is the last element, 1. After pivoting, we have the following arrays:

+
   less than pivot: [ ]
+    equal to pivot: [ 1 ]
+greater than pivot: [ 7, 6, 5, 4, 3, 2 ]
+
+

Now recursively partition the "greater than" subarray and get:

+
   less than pivot: [ ]
+    equal to pivot: [ 2 ]
+greater than pivot: [ 7, 6, 5, 4, 3 ]
+
+

And again:

+
   less than pivot: [ ]
+    equal to pivot: [ 3 ]
+greater than pivot: [ 7, 6, 5, 4 ]
+
+

And so on...

+

That's no good, because this pretty much reduces quicksort to the much slower insertion sort. For quicksort to be efficient, it needs to split the array into roughly two halves.

+

The optimal pivot for this example would have been 4, so we'd get:

+
   less than pivot: [ 3, 2, 1 ]
+    equal to pivot: [ 4 ]
+greater than pivot: [ 7, 6, 5 ]
+
+

You might think this means we should always choose the middle element rather than the first or the last, but imagine what happens in the following situation:

+
[ 7, 6, 5, 1, 4, 3, 2 ]
+
+

Now the middle element is 1 and that gives the same lousy results as in the previous example.

+

Ideally, the pivot is the median element of the array that you're partitioning, i.e. the element that sits in the middle of the sorted array. Of course, you won't know what the median is until after you've sorted the array, so this is a bit of a chicken-and-egg problem. However, there are some tricks to choose good, if not ideal, pivots.

+

One trick is "median-of-three", where you find the median of the first, middle, and last element in this subarray. In theory that often gives a good approximation of the true median.

+

Another common solution is to choose the pivot randomly. Sometimes this may result in choosing a suboptimal pivot but on average this gives very good results.

+

Here is how you can do quicksort with a randomly chosen pivot:

+
func quicksortRandom<T: Comparable>(_ a: inout [T], low: Int, high: Int) {
+  if low < high {
+    let pivotIndex = random(min: low, max: high)         // 1
+
+    (a[pivotIndex], a[high]) = (a[high], a[pivotIndex])  // 2
+
+    let p = partitionLomuto(&a, low: low, high: high)
+    quicksortRandom(&a, low: low, high: p - 1)
+    quicksortRandom(&a, low: p + 1, high: high)
+  }
+}
+

There are two important differences with before:

+
    +
  1. +

    The random(min:max:) function returns an integer in the range min...max, inclusive. This is our pivot index.

    +
    2. +
  2. +

    Because the Lomuto scheme expects a[high] to be the pivot entry, we swap a[pivotIndex] with a[high] to put the pivot element at the end before calling partitionLomuto().

    +
    3. +
+

It may seem strange to use random numbers in something like a sorting function, but it is necessary to make quicksort behave efficiently under all circumstances. With bad pivots, the performance of quicksort can be quite horrible, O(n^2). But if you choose good pivots on average, for example by using a random number generator, the expected running time becomes O(n log n), which is as good as sorting algorithms get.

+

Dutch national flag partitioning

+

But there are more improvements to make! In the first example of quicksort I showed you, we ended up with an array that was partitioned like this:

+
[ values < pivot | values equal to pivot | values > pivot ]
+
+

But as you've seen with the Lomuto partitioning scheme, if the pivot occurs more than once the duplicates end up in the left half. And with Hoare's scheme the pivot can be all over the place. The solution to this is "Dutch national flag" partitioning, named after the fact that the Dutch flag has three bands just like we want to have three partitions.

+

The code for this scheme is:

+
func partitionDutchFlag<T: Comparable>(_ a: inout [T], low: Int, high: Int, pivotIndex: Int) -> (Int, Int) {
+  let pivot = a[pivotIndex]
+
+  var smaller = low
+  var equal = low
+  var larger = high
+
+  while equal <= larger {
+    if a[equal] < pivot {
+      swap(&a, smaller, equal)
+      smaller += 1
+      equal += 1
+    } else if a[equal] == pivot {
+      equal += 1
+    } else {
+      swap(&a, equal, larger)
+      larger -= 1
+    }
+  }
+  return (smaller, larger)
+}
+

This works very similarly to the Lomuto scheme, except that the loop partitions the array into four (possibly empty) regions:

+ +

Note that this doesn't assume the pivot is in a[high]. Instead, you have to pass in the index of the element you wish to use as a pivot.

+

An example of how to use it:

+
var list = [ 10, 0, 3, 9, 2, 14, 8, 27, 1, 5, 8, -1, 26 ]
+partitionDutchFlag(&list, low: 0, high: list.count - 1, pivotIndex: 10)
+list  // show the results
+

Just for fun, we're giving it the index of the other 8 this time. The result is:

+
[ -1, 0, 3, 2, 5, 1, 8, 8, 27, 14, 9, 26, 10 ]
+
+

Notice how the two 8s are in the middle now. The return value from partitionDutchFlag() is a tuple, (6, 7). This is the range that contains the pivot value(s).

+

Here is how you would use it in quicksort:

+
func quicksortDutchFlag<T: Comparable>(_ a: inout [T], low: Int, high: Int) {
+  if low < high {
+    let pivotIndex = random(min: low, max: high)
+    let (p, q) = partitionDutchFlag(&a, low: low, high: high, pivotIndex: pivotIndex)
+    quicksortDutchFlag(&a, low: low, high: p - 1)
+    quicksortDutchFlag(&a, low: q + 1, high: high)
+  }
+}
+

Using Dutch flag partitioning makes for a more efficient quicksort if the array contains many duplicate elements. (And I'm not just saying that because I'm Dutch!)

+
+

Note: The above implementation of partitionDutchFlag() uses a custom swap() routine for swapping the contents of two array elements. Unlike Swift's own swap(), this doesn't give an error when the two indices refer to the same array element. See Quicksort.swift for the code.

+
+

See also

+

Quicksort on Wikipedia

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Rabin-Karp/index.html b/Rabin-Karp/index.html new file mode 100644 index 000000000..8ed53af1a --- /dev/null +++ b/Rabin-Karp/index.html @@ -0,0 +1,67 @@ + + + Rabin-Karp + + + +

Rabin-Karp string search algorithm

+

The Rabin-Karp string search algorithm is used to search text for a pattern.

+

A practical application of the algorithm is detecting plagiarism. Given source material, the algorithm can rapidly search through a paper for instances of sentences from the source material, ignoring details such as case and punctuation. Because of the abundance of the sought strings, single-string searching algorithms are impractical.

+

Example

+

Given a text of "The big dog jumped over the fox" and a search pattern of "ump" this will return 13.
+It starts by hashing "ump" then hashing "The". If hashed don't match then it slides the window a character
+at a time (e.g. "he ") and subtracts out the previous hash from the "T".

+

Algorithm

+

The Rabin-Karp algorithm uses a sliding window the size of the search pattern. It starts by hashing the search pattern, then
+hashing the first x characters of the text string where x is the length of the search pattern. It then slides the window one character over and uses
+the previous hash value to calculate the new hash faster. Only when it finds a hash that matches the hash of the search pattern will it compare
+the two strings it see if they are the same (to prevent a hash collision from producing a false positive).

+

The code

+

The major search method is next. More implementation details are in rabin-karp.swift

+
public func search(text: String , pattern: String) -> Int {
+    // convert to array of ints
+    let patternArray = pattern.flatMap { $0.asInt }
+    let textArray = text.flatMap { $0.asInt }
+
+    if textArray.count < patternArray.count {
+        return -1
+    }
+
+    let patternHash = hash(array: patternArray)
+    var endIdx = patternArray.count - 1
+    let firstChars = Array(textArray[0...endIdx])
+    let firstHash = hash(array: firstChars)
+
+    if (patternHash == firstHash) {
+        // Verify this was not a hash collision
+        if firstChars == patternArray {
+            return 0
+        }
+    }
+
+    var prevHash = firstHash
+    // Now slide the window across the text to be searched
+    for idx in 1...(textArray.count - patternArray.count) {
+        endIdx = idx + (patternArray.count - 1)
+        let window = Array(textArray[idx...endIdx])
+        let windowHash = nextHash(prevHash: prevHash, dropped: textArray[idx - 1], added: textArray[endIdx], patternSize: patternArray.count - 1)
+
+        if windowHash == patternHash {
+            if patternArray == window {
+                return idx
+            }
+        }
+
+        prevHash = windowHash
+    }
+
+    return -1
+}
+

This code can be tested in a playground using the following:

+
  search(text: "The big dog jumped"", "ump")
+

This will return 13 since ump is in the 13 position of the zero based string.

+

Additional Resources

+

Rabin-Karp Wikipedia

+

Written by Bill Barbour

+ + diff --git a/Radix Sort/index.html b/Radix Sort/index.html new file mode 100644 index 000000000..178e60f81 --- /dev/null +++ b/Radix Sort/index.html @@ -0,0 +1,30 @@ + + + Radix Sort + + + +

Radix Sort

+

Radix sort is a sorting algorithm that takes as input an array of integers and uses a sorting subroutine( that is often another efficient sorting algorith) to sort the integers by their radix, or rather their digit. Counting Sort, and Bucket Sort are often times used as the subroutine for Radix Sort.

+

Example

+ +

Step 1:

+

The first step in this algorithm is to define the digit or rather the "base" or radix that we will use to sort.
+For this example we will let radix = 10, since the integers we are working with in the example are of base 10.

+

Step 2:

+

The next step is to simply iterate n times (where n is the number of digits in the largest integer in the input array), and upon each iteration perform a sorting subroutine on the current digit in question.

+

Algorithm in Action

+

Let's take a look at our example input array.

+

The largest integer in our array is 802, and it has three digits (ones, tens, hundreds). So our algorithm will iterate three times whilst performing some sorting algorithm on the digits of each integer.

+ +

See also Wikipedia.

+

Written for the Swift Algorithm Club by Christian Encarnacion

+ + diff --git a/Radix Tree/index.html b/Radix Tree/index.html new file mode 100644 index 000000000..da39b5bcf --- /dev/null +++ b/Radix Tree/index.html @@ -0,0 +1,49 @@ + + + Radix Tree + + + +

Radix Tree

+

A radix tree is a tree where a node can have any number of children. Each edge leading from a node to a child has a label (usually a string). A radix tree is often used to store strings or IP addresses. By traversing from the root to any leaf in the tree, concatenating all the labels of edges along the way, you can find any string.

+

This is an example of a radix tree (image from wikipedia.org):

+

+

Implemenation

+

The radix tree is represented by a RadixTree object. This class has one member variable, root, of type Root. The Root is the object at the top of every RadixTree.

+

Every other node is of type Edge and is a child (or child of a child, etc.) of the root.

+

The difference between Edge and Root is that edges have a label (of type String) and a reference to a parent node.

+

This approach is a little different from the conventional way of creating tree data structures but is easier to wrap your head around (see the picture above).

+

Operations

+

Typical operations on a radix tree are:

+ +

The running time of lookup, insertion, and deletion is O(k) where k is the length of the key. This is different from most trees because these operations usually run in O(log n) time where n is the number of nodes in the tree.

+

Lookup

+

The find() function returns true if the string you are searching for is in the tree and false if it is not.

+

Every RadixTree contains at the very least the empty string "" so find("") will always return true.

+

A string is considered "in the tree" if the text you are searching for is a concatenation of Edge labels while traversing downwards, or a prefix of that concatenation.

+

For example, if you look at the example tree above, find("roma") will return true because it is a prefix of the traversal "r" -> "om" -> "an".

+

On the other hand, find("romans") will return false.

+

Insertion

+

The insert() function lets you add new strings to the radix tree.

+

This function returns true if the string you are trying to insert was successfully inserted into the RadixTree and false if the string is already in the tree.

+

Deletion

+

The remove() removes a string from the tree. When a string is removed, any other strings that have a prefix of the removed string are removed as well.

+

For example, remove("rom") will also remove "roman", "rome", and "romulus" if those strings are in the tree as well. Calling remove("") will remove all strings in the tree.

+

printTree()

+

You can use the printTree() function to visualize the tree. This function is a little buggy and not perfect yet but gets the job done thus far.

+

Helper functions

+

The sharedPrefix() function is a non-member function in the RadixTree.swift file. It takes in two String objects and returns the shared prefix between them.

+

For example, sharedPrefix("apples", "oranges") will return "", and sharedPrefix("court", "coral") will return "co".

+

This function is used in the implementation of the find(), insert(), and remove() functions.

+

See also

+

Radix Tree - Wikipedia

+

Written for Swift Algorithm Club by Steven Scally

+ + diff --git a/Red-Black Tree/index.html b/Red-Black Tree/index.html new file mode 100644 index 000000000..250f983dd --- /dev/null +++ b/Red-Black Tree/index.html @@ -0,0 +1,177 @@ + + + Red-Black Tree + + + +

Red-Black Tree

+

A red-black tree (RBT) is a balanced version of a Binary Search Tree guaranteeing that the basic operations (search, predecessor, successor, minimum, maximum, insert and delete) have a logarithmic worst case performance.

+

Binary search trees (BSTs) have the disadvantage that they can become unbalanced after some insert or delete operations. In the worst case, this could lead to a tree where the nodes build a linked list as shown in the following example:

+
a
+  \
+   b
+    \
+     c
+      \
+       d
+
+

To prevent this issue, RBTs perform rebalancing operations after an insert or delete and store an additional color property at each node which can either be red or black. After each operation a RBT satisfies the following properties:

+

Properties

+
    +
  1. Every node is either red or black
    2. +
  2. The root is black
    3. +
  3. Every leaf (nullLeaf) is black
    4. +
  4. If a node is red, then both its children are black
    5. +
  5. For each node, all paths from the node to descendant leaves contain the same number of black nodes
    6. +
+

Property 5 includes the definition of the black-height of a node x, bh(x), which is the number of black nodes on a path from this node down to a leaf not counting the node itself.
+From [CLRS]

+

Methods

+

Nodes:

+ +

The rotation, insertion and deletion algorithms are implemented based on the pseudo-code provided in [CLRS]

+

Implementation Details

+

For convenience, all nil-pointers to children or the parent (except the parent of the root) of a node are exchanged with a nullLeaf. This is an ordinary node object, like all other nodes in the tree, but with a black color, and in this case a nil value for its children, parent and key. Therefore, an empty tree consists of exactly one nullLeaf at the root.

+

Rotation

+

Left rotation (around x):
+Assumes that x.rightChild y is not a nullLeaf, rotates around the link from x to y, makes y the new root of the subtree with x as y's left child and y's left child as x's right child, where n = a node, [n] = a subtree

+
      |                |
+      x                y
+    /   \     ~>     /   \
+  [A]    y          x    [C]
+        / \        / \
+      [B] [C]    [A] [B]
+
+

Right rotation (around y):
+Assumes that y.leftChild x is not a nullLeaf, rotates around the link from y to x, makes x the new root of the subtree with y as x's right child and x's right child as y's left child, where n = a node, [n] = a subtree

+
      |                |
+      x                y
+    /   \     <~     /   \
+  [A]    y          x    [C]
+        / \        / \
+      [B] [C]    [A] [B]
+
+

As rotation is a local operation only exchanging pointers it's runtime is O(1).

+

Insertion

+

We create a node with the given key and set its color to red. Then we insert it into the the tree by performing a standard insert into a BST. After this, the tree might not be a valid RBT anymore, so we fix the red-black properties by calling the insertFixup algorithm.
+The only violation of the red-black properties occurs at the inserted node z and its parent. Either both are red, or the parent does not exist (so there is a violation since the root must be black).
+We have three distinct cases:
+Case 1: The uncle of z is red. If z.parent is left child, z's uncle is z.grandparent's right child. If this is the case, we recolor and move z to z.grandparent, then we check again for this new z. In the following cases, we denote a red node with (x) and a black node with {x}, p = parent, gp = grandparent and u = uncle

+
         |                   |
+        {gp}               (newZ)
+       /    \     ~>       /    \
+    (p)     (u)         {p}     {u}
+    / \     / \         / \     / \
+  (z)  [C] [D] [E]    (z) [C] [D] [E]
+  / \                 / \
+[A] [B]             [A] [B]
+
+
+

Case 2a: The uncle of z is black and z is right child. Here, we move z upwards, so z's parent is the newZ and then we rotate around this newZ. After this, we have Case 2b.

+
         |                   |
+        {gp}                {gp}
+       /    \     ~>       /    \
+    (p)      {u}         (z)     {u}
+    / \      / \         / \     / \
+  [A]  (z)  [D] [E]  (newZ) [C] [D] [E]
+       / \            / \
+     [B] [C]        [A] [B]
+
+
+

Case 2b: The uncle of z is black and z is left child. In this case, we recolor z.parent to black and z.grandparent to red. Then we rotate around z's grandparent. Afterwards we have valid red-black tree.

+
         |                   |
+        {gp}                {p}
+       /    \     ~>       /    \
+    (p)      {u}         (z)    (gp)
+    / \      / \         / \     / \
+  (z)  [C] [D] [E]    [A] [B]  [C]  {u}
+  / \                               / \
+[A] [B]                           [D] [E]
+
+
+

Running time of this algorithm:

+ +

Deletion

+

We search for the node with the given key, and if it exists we delete it by performing a standard delete from a BST. If the deleted nodes' color was red everything is fine, otherwise red-black properties may be violated so we call the fixup procedure deleteFixup.
+Violations can be that the parent and child of the deleted node x where red, so we now have two adjacent red nodes, or if we deleted the root, the root could now be red, or the black height property is violated.
+We have 4 cases: We call deleteFixup on node x
+Case 1: The sibling of x is red. The sibling is the other child of x's parent, which is not x itself. In this case, we recolor the parent of x and x.sibling then we left rotate around x's parent. In the following cases s = sibling of x, (x) = red, {x} = black, |x| = red/black.

+
        |                   |
+       {p}                 {s}
+      /    \     ~>       /    \
+    {x}    (s)         (p)     [D]
+    / \    / \         / \     
+  [A] [B] [C] [D]    {x} [C]
+                     / \  
+                   [A] [B]
+
+
+

Case 2: The sibling of x is black and has two black children. Here, we recolor x.sibling to red, move x upwards to x.parent and check again for this newX.

+
        |                    |
+       |p|                 |newX|
+      /    \     ~>       /     \
+    {x}     {s}          {x}     (s)
+    / \    /   \          / \   /   \
+  [A] [B] {l}  {r}     [A] [B] {l}  {r}
+          / \  / \             / \  / \
+        [C][D][E][F]         [C][D][E][F]
+
+
+

Case 3: The sibling of x is black with one black child to the right. In this case, we recolor the sibling to red and sibling.leftChild to black, then we right rotate around the sibling. After this we have case 4.

+
        |                    |
+       |p|                  |p|
+      /    \     ~>       /     \
+    {x}     {s}          {x}     {l}
+    / \    /   \         / \    /   \
+  [A] [B] (l)  {r}     [A] [B] [C]  (s)
+          / \  / \                  / \
+        [C][D][E][F]               [D]{e}
+                                      / \
+                                    [E] [F]
+
+
+

Case 4: The sibling of x is black with one red child to the right. Here, we recolor the sibling to the color of x.parent and x.parent and sibling.rightChild to black. Then we left rotate around x.parent. After this operation we have a valid red-black tree. Here, ||x|| denotes that x can have either color red or black, but that this can be different to |x| color. This is important, as s gets the same color as p.

+
        |                        |
+       ||p||                   ||s||
+      /    \     ~>           /     \
+    {x}     {s}              {p}     {r}
+    / \    /   \            /  \     /  \
+  [A] [B] |l|  (r)        {x}  |l|  [E] [F]
+          / \  / \        / \  / \
+        [C][D][E][F]    [A][B][C][D]
+
+
+

Running time of this algorithm:

+ +

Resources:

+

[CLRS] T. Cormen, C. Leiserson, R. Rivest, and C. Stein. "Introduction to Algorithms", Third Edition. 2009

+

Written for Swift Algorithm Club by Ute Schiehlen. Updated from Jaap Wijnen and Ashwin Raghuraman's contributions. Swift 4.2 check by Bruno Scheele.

+ + diff --git a/Ring Buffer/index.html b/Ring Buffer/index.html new file mode 100644 index 000000000..0aad0b8be --- /dev/null +++ b/Ring Buffer/index.html @@ -0,0 +1,132 @@ + + + Ring Buffer + + + +

Ring Buffer

+

Also known as a circular buffer.

+

The problem with a queue based on an array is that adding new items to the back of the queue is fast, O(1), but removing items from the front of the queue is slow, O(n). Removing is slow because it requires the remaining array elements to be shifted in memory.

+

A more efficient way to implement a queue is to use a ring buffer or circular buffer. This is an array that conceptually wraps around back to the beginning, so you never have to remove any items. All operations are O(1).

+

Here is how it works in principle. We have an array of a fixed size, say 5 items:

+
[    ,    ,    ,    ,     ]
+ r
+ w
+
+

Initially, the array is empty and the read (r) and write (w) pointers are at the beginning.

+

Let's add some data to this array. We'll write, or "enqueue", the number 123:

+
[ 123,    ,    ,    ,     ]
+  r
+  ---> w
+
+

Each time we add data, the write pointer moves one step forward. Let's add a few more elements:

+
[ 123, 456, 789, 666,     ]
+  r    
+       -------------> w
+
+

There is now one open spot left in the array, but rather than enqueuing another item the app decides to read some data. That's possible because the write pointer is ahead of the read pointer, meaning data is available for reading. The read pointer advances as we read the available data:

+
[ 123, 456, 789, 666,     ]
+  ---> r              w
+
+

Let's read two more items:

+
[ 123, 456, 789, 666,     ]
+       --------> r    w
+
+

Now the app decides it's time to write again and enqueues two more data items, 333 and 555:

+
[ 123, 456, 789, 666, 333 ]
+                 r    ---> w
+
+

Whoops, the write pointer has reached the end of the array, so there is no more room for object 555. What now? Well, this is why it's a circular buffer: we wrap the write pointer back to the beginning and write the remaining data:

+
[ 555, 456, 789, 666, 333 ]
+  ---> w         r        
+
+

We can now read the remaining three items, 666, 333, and 555.

+
[ 555, 456, 789, 666, 333 ]
+       w         --------> r        
+
+

Of course, as the read pointer reaches the end of the buffer it also wraps around:

+
[ 555, 456, 789, 666, 333 ]
+       w            
+  ---> r
+
+

And now the buffer is empty again because the read pointer has caught up with the write pointer.

+

Here is a very basic implementation in Swift:

+
public struct RingBuffer<T> {
+  fileprivate var array: [T?]
+  fileprivate var readIndex = 0
+  fileprivate var writeIndex = 0
+
+  public init(count: Int) {
+    array = [T?](repeating: nil, count: count)
+  }
+
+  public mutating func write(_ element: T) -> Bool {
+    if !isFull {
+      array[writeIndex % array.count] = element
+      writeIndex += 1
+      return true
+    } else {
+      return false
+    }
+  }
+
+  public mutating func read() -> T? {
+    if !isEmpty {
+      let element = array[readIndex % array.count]
+      readIndex += 1
+      return element
+    } else {
+      return nil
+    }
+  }
+
+  fileprivate var availableSpaceForReading: Int {
+    return writeIndex - readIndex
+  }
+
+  public var isEmpty: Bool {
+    return availableSpaceForReading == 0
+  }
+
+  fileprivate var availableSpaceForWriting: Int {
+    return array.count - availableSpaceForReading
+  }
+
+  public var isFull: Bool {
+    return availableSpaceForWriting == 0
+  }
+}
+

The RingBuffer object has an array for the actual storage of the data, and readIndex and writeIndex variables for the "pointers" into the array. The write() function puts the new element into the array at the writeIndex, and the read() function returns the element at the readIndex.

+

But hold up, you say, how does this wrapping around work? There are several ways to accomplish this and I chose a slightly controversial one. In this implementation, the writeIndex and readIndex always increment and never actually wrap around. Instead, we do the following to find the actual index into the array:

+
array[writeIndex % array.count]
+

and:

+
array[readIndex % array.count]
+

In other words, we take the modulo (or the remainder) of the read index and write index divided by the size of the underlying array.

+

The reason this is a bit controversial is that writeIndex and readIndex always increment, so in theory these values could become too large to fit into an integer and the app will crash. However, a quick back-of-the-napkin calculation should take away those fears.

+

Both writeIndex and readIndex are 64-bit integers. If we assume that they are incremented 1000 times per second, which is a lot, then doing this continuously for one year requires ceil(log_2(365 * 24 * 60 * 60 * 1000)) = 35 bits. That leaves 28 bits to spare, so that should give you about 2^28 years before running into problems. That's a long time. :-)

+

To play with this ring buffer, copy the code to a playground and do the following to mimic the example above:

+
var buffer = RingBuffer<Int>(count: 5)
+
+buffer.write(123)
+buffer.write(456)
+buffer.write(789)
+buffer.write(666)
+
+buffer.read()   // 123
+buffer.read()   // 456
+buffer.read()   // 789
+
+buffer.write(333)
+buffer.write(555)
+
+buffer.read()   // 666
+buffer.read()   // 333
+buffer.read()   // 555
+buffer.read()   // nil
+

You've seen that a ring buffer can make a more optimal queue but it also has a disadvantage: the wrapping makes it tricky to resize the queue. But if a fixed-size queue is suitable for your purposes, then you're golden.

+

A ring buffer is also very useful for when a producer of data writes into the array at a different rate than the consumer of the data reads it. This happens often with file or network I/O. Ring buffers are also the preferred way of communicating between high priority threads (such as an audio rendering callback) and other, slower, parts of the system.

+

The implementation given here is not thread-safe. It only serves as an example of how a ring buffer works. That said, it should be fairly straightforward to make it thread-safe for a single reader and single writer by using OSAtomicIncrement64() to change the read and write pointers.

+

A cool trick to make a really fast ring buffer is to use the operating system's virtual memory system to map the same buffer onto different memory pages. Crazy stuff but worth looking into if you need to use a ring buffer in a high performance environment.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Rootish Array Stack/index.html b/Rootish Array Stack/index.html new file mode 100644 index 000000000..b65e8ad01 --- /dev/null +++ b/Rootish Array Stack/index.html @@ -0,0 +1,217 @@ + + + Rootish Array Stack + + + +

Rootish Array Stack

+

A Rootish Array Stack is an ordered array based structure that minimizes wasted space (based on Gauss's summation technique). A Rootish Array Stack consists of an array holding many fixed size arrays in ascending size.

+

Rootish Array Stack Intro

+

A resizable array holds references to blocks (arrays of fixed size). A block's capacity is the same as it's index in the resizable array. Blocks don't grow/shrink like regular Swift arrays. Instead, when their capacity is reached, a new slightly larger block is created. When a block is emptied the last block is freed. This is a great improvement on what Swift arrays do in terms of wasted space.

+

Rootish Array Stack Intro

+

Here you can see how insert/remove operations would behave (very similar to how a Swift array handles such operations).

+

Gauss' Summation Trick

+

One of the most well known legends about famous mathematician Carl Friedrich Gauss goes back to when he was in primary school. One day, Gauss' teacher asked his class to add up all the numbers from 1 to 100, hoping that the task would take long enough for him to step out for a smoke break. The teacher was shocked when young Gauss had his hand up with the answer 5050. So soon? The teacher suspected a cheat, but no. Gauss had found a formula to sidestep the problem of manually adding up all the numbers 1 by 1. His formula:

+
sum from 1...n = n * (n + 1) / 2
+
+

To understand this imagine n blocks where x represents 1 unit. In this example let n be 5:

+
blocks:     [x] [x x] [x x x] [x x x x] [x x x x x]
+# of x's:    1    2      3        4          5
+
+

Block 1 has 1 x, block 2 as 2 xs, block 3 has 3 xs, etc...

+

If you wanted to take the sum of all the blocks from 1 to n, you could go through and count them one by one. This is okay, but for a large sequence of blocks that could take a long time! Instead, you could arrange the blocks to look like a half pyramid:

+
# |  blocks
+--|-------------
+1 |  x
+2 |  x x
+3 |  x x x
+4 |  x x x x
+5 |  x x x x x
+
+
+

Then we mirror the half pyramid and rearrange the image so that it fits with the original half pyramid in a rectangular shape:

+
x                  o      x o o o o o
+x x              o o      x x o o o o
+x x x          o o o  =>  x x x o o o
+x x x x      o o o o      x x x x o o
+x x x x x  o o o o o      x x x x x o
+
+

Here we have n rows and n + 1 columns. 5 rows and 6 columns.

+

We can calculate the sum just as we would an area! Let's also express the width and height in terms of n:

+
area of a rectangle = height * width = n * (n + 1)
+
+

We only want to calculate the amount of xs, not the amount of os. Since there's a 1:1 ratio between xs and os we can just divide our area by 2!

+
area of only x = n * (n + 1) / 2
+
+

Voila! A super fast way to take a sum of all the blocks! This equation is useful for deriving fast block and inner block index equations.

+ +

Get/Set with Speed

+

Next, we want to find an efficient and accurate way to access an element at a random index. For example, which block does rootishArrayStack[12] point to? To answer this we will need more math!
+Determining the inner block index turns out to be easy. If index is in some block then:

+
inner block index = index - block * (block + 1) / 2
+
+

Determining which block an index points to is more difficult. The number of elements up to and including the element requested is: index + 1 elements. The number of elements in blocks 0...block is (block + 1) * (block + 2) / 2 (equation derived above). The relationship between the block and the index is as follows:

+
(block + 1) * (block + 2) / 2 >= index + 1
+
+

This can be rewritten as:

+
(block)^2 + (3 * block) - (2 * index) >= 0
+
+

Using the quadratic formula we get:

+
block = (-3 ± √(9 + 8 * index)) / 2
+
+

A negative block doesn't make sense, so we take the positive root instead. In general, this solution is not an integer. However, going back to our inequality, we want the smallest block such that block => (-3 + √(9 + 8 * index)) / 2. Next, we take the ceiling of the result:

+
block = ⌈(-3 + √(9 + 8 * index)) / 2⌉
+
+

Now we can figure out what rootishArrayStack[12] points to! First, let's see which block the 12 points to:

+
block = ⌈(-3 + √(9 + 8 * (12))) / 2⌉
+block = ⌈(-3 + √105) / 2⌉
+block = ⌈(-3 + (10.246950766)) / 2⌉
+block = ⌈(7.246950766) / 2⌉
+block = ⌈3.623475383⌉
+block = 4
+
+

Next lets see which innerBlockIndex 12 points to:

+
inner block index = (12) - (4) * ((4) + 1) / 2
+inner block index = (12) - (4) * (5) / 2
+inner block index = (12) - 10
+inner block index = 2
+
+

Therefore, rootishArrayStack[12] points to the block at index 4 and at inner block index 2.
+Rootish Array Stack Intro

+

Interesting Discovery

+

Using the block equation, we can see that the number of blocks is proportional to the square root of the number of elements: O(blocks) = O(√n).

+

Implementation Details

+

Let's start with instance variables and struct declaration:

+
import Darwin
+
+public struct RootishArrayStack<T> {
+
+  fileprivate var blocks = [Array<T?>]()
+  fileprivate var internalCount = 0
+
+  public init() { }
+
+  var count: Int {
+    return internalCount
+  }
+
+  ...
+
+}
+
+

The elements are of generic type T, so data of any kind can be stored in the list. blocks will be a resizable array to hold fixed sized arrays that take type T?.

+
+

The reason for the fixed size arrays taking type T? is so that references to elements aren't retained after they've been removed. Eg: if you remove the last element, the last index must be set to nil to prevent the last element being held in memory at an inaccessible index.

+
+

internalCount is an internal mutable counter that keeps track of the number of elements. count is a read only variable that returns the internalCount value. Darwin is imported here to provide simple math functions such as ceil() and sqrt().

+

The capacity of the structure is simply the Gaussian summation trick:

+
var capacity: Int {
+  return blocks.count * (blocks.count + 1) / 2
+}
+

Next, let's look at how we would get and set elements:

+
fileprivate func block(fromIndex: Int) -> Int {
+  let block = Int(ceil((-3.0 + sqrt(9.0 + 8.0 * Double(index))) / 2))
+  return block
+}
+
+fileprivate func innerBlockIndex(fromIndex index: Int, fromBlock block: Int) -> Int {
+  return index - block * (block + 1) / 2
+}
+
+public subscript(index: Int) -> T {
+  get {
+    let block = self.block(fromIndex: index)
+    let innerBlockIndex = self.innerBlockIndex(fromIndex: index, fromBlock: block)
+    return blocks[block][innerBlockIndex]!
+  }
+  set(newValue) {
+    let block = self.block(fromIndex: index)
+    let innerBlockIndex = self.innerBlockIndex(fromIndex: index, fromBlock: block)
+    blocks[block][innerBlockIndex] = newValue
+  }
+}
+

block(fromIndex:) and innerBlockIndex(fromIndex:, fromBlock:) are wrapping the block and inner block index equations we derived earlier. superscript lets us have get and set access to the structure with the familiar [index:] syntax. For both get and set in superscript we use the same logic:

+
    +
  1. determine the block that the index points to
    2. +
  2. determine the inner block index
    3. +
  3. get/set the value
    4. +
+

Next, let's look at how we would growIfNeeded() and shrinkIfNeeded().

+
fileprivate mutating func growIfNeeded() {
+  if capacity - blocks.count < count + 1 {
+    let newArray = [T?](repeating: nil, count: blocks.count + 1)
+    blocks.append(newArray)
+  }
+}
+
+fileprivate mutating func shrinkIfNeeded() {
+  if capacity + blocks.count >= count {
+    while blocks.count > 0 && (blocks.count - 2) * (blocks.count - 1) / 2 >    count {
+      blocks.remove(at: blocks.count - 1)
+    }
+  }
+}
+

If our data set grows or shrinks in size, we want our data structure to accommodate the change.
+Just like a Swift array, when a capacity threshold is met we will grow or shrink the size of our structure. For the Rootish Array Stack we want to grow if the second last block is full on an insert operation, and shrink if the two last blocks are empty.

+

Now to the more familiar Swift array behaviour.

+
public mutating func insert(element: T, atIndex index: Int) {
+	growIfNeeded()
+	internalCount += 1
+	var i = count - 1
+	while i > index {
+		self[i] = self[i - 1]
+		i -= 1
+	}
+	self[index] = element
+}
+
+public mutating func append(element: T) {
+	insert(element: element, atIndex: count)
+}
+
+public mutating func remove(atIndex index: Int) -> T {
+	let element = self[index]
+	for i in index..<count - 1 {
+		self[i] = self[i + 1]
+	}
+	internalCount -= 1
+	makeNil(atIndex: count)
+	shrinkIfNeeded()
+	return element
+}
+
+fileprivate mutating func makeNil(atIndex index: Int) {
+  let block = self.block(fromIndex: index)
+  let innerBlockIndex = self.innerBlockIndex(fromIndex: index, fromBlock: block)
+  blocks[block][innerBlockIndex] = nil
+}
+

To insert(element:, atIndex:) we move all elements after the index to the right by 1. After space has been made for the element, we set the value using the subscript convenience method.
+append(element:) is just a convenience method to insert to the end.
+To remove(atIndex:) we move all the elements after the index to the left by 1. After the removed value is covered by it's proceeding value, we set the last value in the structure to nil.
+makeNil(atIndex:) uses the same logic as our subscript method but is used to set the root optional at a particular index to nil (because setting it's wrapped value to nil is something only the user of the data structure should do).

+
+

Setting a optionals value to nil is different than setting it's wrapped value to nil. An optionals wrapped value is an embedded type within the optional reference. This means that a nil wrapped value is actually .some(.none) whereas setting the root reference to nil is .none. To better understand Swift optionals I recommend checking out @SebastianBoldt's article Swift! Optionals?.

+
+

Performance

+ +

Analysis of Growing and Shrinking

+

The performance analysis doesn't account for the cost to grow and shrink. Unlike a regular Swift array, grow and shrink operations don't copy all the elements into a backing array. They only allocate or free an array proportional to the number of blocks. The number of blocks is proportional to the square root of the number of elements. Growing and shrinking only costs O(√n).

+

Wasted Space

+

Wasted space is how much memory with respect to the number of elements n is unused. The Rootish Array Stack never has more than 2 empty blocks and it never has less than 1 empty block. The last two blocks are proportional to the number of blocks, which is proportional to the square root of the number of elements. The number of references needed to point to each block is the same as the number of blocks. Therefore, the amount of wasted space with respect to the number of elements is O(√n).

+

Written for Swift Algorithm Club by @BenEmdon

+

With help from OpenDataStructures.org

+ + diff --git a/Run-Length Encoding/index.html b/Run-Length Encoding/index.html new file mode 100644 index 000000000..522611509 --- /dev/null +++ b/Run-Length Encoding/index.html @@ -0,0 +1,139 @@ + + + Run-Length Encoding + + + +

Run-Length Encoding (RLE)

+

RLE is probably the simplest way to do compression. Let's say you have data that looks like this:

+
aaaaabbbcdeeeeeeef...
+
+

then RLE encodes it as follows:

+
5a3b1c1d7e1f...
+
+

Instead of repeating bytes, you first write how often that byte occurs and then the byte's actual value. So 5a means aaaaa. If the data has a lot of "byte runs", that is lots of repeating bytes, then RLE can save quite a bit of space. It works quite well on images.

+

There are many different ways you can implement RLE. Here's an extension of Data that does a version of RLE inspired by the old PCX image file format.

+

The rules are these:

+ +

Here is the compression code. It returns a new Data object containing the run-length encoded bytes:

+
extension Data {
+    public func compressRLE() -> Data {
+        var data = Data()
+        self.withUnsafeBytes { (uPtr: UnsafePointer<UInt8>) in
+            var ptr = uPtr
+            let end = ptr + count
+            while ptr < end { 						//1
+                var count = 0
+                var byte = ptr.pointee
+                var next = byte
+
+                while next == byte && ptr < end && count < 64 { //2
+                    ptr = ptr.advanced(by: 1)
+                    next = ptr.pointee
+                    count += 1
+                }
+
+                if count > 1 || byte >= 192 {       // 3
+                    var size = 191 + UInt8(count)
+                    data.append(&size, count: 1)
+                    data.append(&byte, count: 1)
+                } else {                            // 4
+                    data.append(&byte, count: 1)
+                }
+            }
+        }
+        return data
+    }
+ }
+

How it works:

+
    +
  1. +

    We use an UnsafePointer to step through the bytes of the original Data object.

    +
    2. +
  2. +

    At this point we've read the current byte value into the byte variable. If the next byte is the same, then we keep reading until we find a byte value that is different, or we reach the end of the data. We also stop if the run is 64 bytes because that's the maximum we can encode.

    +
    3. +
  3. +

    Here, we have to decide how to encode the bytes we just read. The first possibility is that we've read a run of 2 or more bytes (up to 64). In that case we write out two bytes: the length of the run followed by the byte value. But it's also possible we've read a single byte with a value >= 192. That will also be encoded with two bytes.

    +
    4. +
  4. +

    The third possibility is that we've read a single byte < 192. That simply gets copied to the output verbatim.

    +
    5. +
+

You can test it like this in a playground:

+
let originalString = "aaaaabbbcdeeeeeeef"
+let utf8 = originalString.data(using: String.Encoding.utf8)!
+let compressed = utf8.compressRLE()
+

The compressed Data object should be <c461c262 6364c665 66>. Let's decode that by hand to see what has happened:

+
c4    This is 196 in decimal. It means the next byte appears 5 times.
+61    The data byte "a".
+c2    The next byte appears 3 times.
+62    The data byte "b".
+63    The data byte "c". Because this is < 192, it's a single data byte.
+64    The data byte "d". Also appears just once.
+c6    The next byte will appear 7 times.
+65    The data byte "e".
+66    The data byte "f". Appears just once.
+
+

So that's 9 bytes encoded versus 18 original. That's a savings of 50%. Of course, this was only a simple test case... If you get unlucky and there are no byte runs at all in your original data, then this method will actually make the encoded data twice as large! So it really depends on the input data.

+

Here is the decompression code:

+
public func decompressRLE() -> Data {
+        var data = Data()
+        self.withUnsafeBytes { (uPtr: UnsafePointer<UInt8>) in
+            var ptr = uPtr
+            let end = ptr + count
+
+            while ptr < end {
+                // Read the next byte. This is either a single value less than 192,
+                // or the start of a byte run.
+                var byte = ptr.pointee							// 1
+                ptr = ptr.advanced(by: 1)
+
+                if byte < 192 {                     // 2
+                    data.append(&byte, count: 1)
+                } else if ptr < end {               // 3
+                    // Read the actual data value.
+                    var value = ptr.pointee
+                    ptr = ptr.advanced(by: 1)
+
+                    // And write it out repeatedly.
+                    for _ in 0 ..< byte - 191 {
+                        data.append(&value, count: 1)
+                    }
+                }
+            }
+        }
+        return data
+    }
+
+
    +
  1. +

    Again this uses an UnsafePointer to read the Data. Here we read the next byte; this is either a single value less than 192, or the start of a byte run.

    +
    2. +
  2. +

    If it's a single value, then it's just a matter of copying it to the output.

    +
    3. +
  3. +

    But if the byte is the start of a run, we have to first read the actual data value and then write it out repeatedly.

    +
    4. +
+

To turn the compressed data back into the original, you'd do:

+
let decompressed = compressed.decompressRLE()
+let restoredString = String(data: decompressed, encoding: NSUTF8StringEncoding)
+

And now originalString == restoredString must be true!

+

Footnote: The original PCX implementation is slightly different. There, a byte value of 192 (0xC0) means that the following byte will be repeated 0 times. This also limits the maximum run size to 63 bytes. Because it makes no sense to store bytes that don't occur, in my implementation 192 means the next byte appears once, and the maximum run length is 64 bytes.

+

This was probably a trade-off when they designed the PCX format way back when. If you look at it in binary, the upper two bits indicate whether a byte is compressed. (If both bits are set then the byte value is 192 or more.) To get the run length you can simply do byte & 0x3F, giving you a value in the range 0 to 63.

+

Written for Swift Algorithm Club by Matthijs Hollemans
+Migrated to Swift3 by Jaap Wijnen

+ + diff --git a/Segment Tree/index.html b/Segment Tree/index.html new file mode 100644 index 000000000..9cbf081cb --- /dev/null +++ b/Segment Tree/index.html @@ -0,0 +1,167 @@ + + + Segment Tree + + + +

Segment Tree

+
+

For an example on lazy propagation, see this article.

+
+

I'm pleased to present to you Segment Tree. It's actually one of my favorite data structures because it's very flexible and simple in realization.

+

Let's suppose that you have an array a of some type and some associative function f. For example, the function can be sum, multiplication, min, max, gcd, and so on.

+

Your task is to:

+ +

For example, if we have an array of numbers:

+
var a = [ 20, 3, -1, 101, 14, 29, 5, 61, 99 ]
+

We want to query this array on the interval from 3 to 7 for the function "sum". That means we do the following:

+
101 + 14 + 29 + 5 + 61 = 210
+
+

because 101 is at index 3 in the array and 61 is at index 7. So we pass all the numbers between 101 and 61 to the sum function, which adds them all up. If we had used the "min" function, the result would have been 5 because that's the smallest number in the interval from 3 to 7.

+

Here's naive approach if our array's type is Int and f is just the sum of two integers:

+
func query(array: [Int], l: Int, r: Int) -> Int {
+  var sum = 0
+  for i in l...r {
+    sum += array[i]
+  }
+  return sum
+}
+

The running time of this algorithm is O(n) in the worst case, that is when l = 0, r = n-1 (where n is the number of elements in the array). And if we have m queries to answer we get O(m*n) complexity.

+

If we have an array with 100,000 items (n = 10^5) and we have to do 100 queries (m = 100), then our algorithm will do 10^7 units of work. Ouch, that doesn't sound very good. Let's look at how we can improve it.

+

Segment trees allow us to answer queries and replace items with O(log n) time. Isn't it magic? ✨

+

The main idea of segment trees is simple: we precalculate some segments in our array and then we can use those without repeating calculations.

+

Structure of segment tree

+

A segment tree is just a binary tree where each node is an instance of the SegmentTree class:

+
public class SegmentTree<T> {
+  private var value: T
+  private var function: (T, T) -> T
+  private var leftBound: Int
+  private var rightBound: Int
+  private var leftChild: SegmentTree<T>?
+  private var rightChild: SegmentTree<T>?
+}
+

Each node has the following data:

+ +

If our array is [1, 2, 3, 4] and the function f = a + b, the segment tree looks like this:

+

structure

+

The leftBound and rightBound of each node are marked in red.

+

Building a segment tree

+

Here's how we create a node of the segment tree:

+
public init(array: [T], leftBound: Int, rightBound: Int, function: @escaping (T, T) -> T) {
+    self.leftBound = leftBound
+    self.rightBound = rightBound
+    self.function = function
+
+    if leftBound == rightBound {                    // 1
+      value = array[leftBound]
+    } else {
+      let middle = (leftBound + rightBound) / 2     // 2
+
+      // 3
+      leftChild = SegmentTree<T>(array: array, leftBound: leftBound, rightBound: middle, function: function)
+      rightChild = SegmentTree<T>(array: array, leftBound: middle+1, rightBound: rightBound, function: function)
+
+      value = function(leftChild!.value, rightChild!.value)  // 4
+    }
+  }
+

Notice that this is a recursive method. You give it an array such as [1, 2, 3, 4] and it builds up the entire tree, from the root node to all the child nodes.

+
    +
  1. +

    The recursion terminates if leftBound and rightBound are equal. Such a SegmentTree instance represents a leaf node. For the input array [1, 2, 3, 4], this process will create four such leaf nodes: 1, 2, 3, and 4. We just fill in the value property with the number from the array.

    +
    2. +
  2. +

    However, if rightBound is still greater than leftBound, we create two child nodes. We divide the current segment into two equal segments (at least, if the length is even; if it's odd, one segment will be slightly larger).

    +
    3. +
  3. +

    Recursively build child nodes for those two segments. The left child node covers the interval [leftBound, middle] and the right child node covers [middle+1, rightBound].

    +
    4. +
  4. +

    After having constructed our child nodes, we can calculate our own value because f(leftBound, rightBound) = f(f(leftBound, middle), f(middle+1, rightBound)). It's math!

    +
    5. +
+

Building the tree is an O(n) operation.

+

Getting answer to query

+

We go through all this trouble so we can efficiently query the tree.

+

Here's the code:

+
  public func query(withLeftBound: leftBound: Int, rightBound: Int) -> T {
+    // 1
+    if self.leftBound == leftBound && self.rightBound == rightBound {
+      return self.value
+    }
+
+    guard let leftChild = leftChild else { fatalError("leftChild should not be nil") }
+    guard let rightChild = rightChild else { fatalError("rightChild should not be nil") }
+
+    // 2
+    if leftChild.rightBound < leftBound {
+      return rightChild.query(withLeftBound: leftBound, rightBound: rightBound)
+
+    // 3
+    } else if rightChild.leftBound > rightBound {
+      return leftChild.query(withLeftBound: leftBound, rightBound: rightBound)
+
+    // 4
+    } else {
+      let leftResult = leftChild.query(withLeftBound: leftBound, rightBound: leftChild.rightBound)
+      let rightResult = rightChild.query(withLeftBound: rightChild.leftBound, rightBound: rightBound)
+      return function(leftResult, rightResult)
+    }
+  }
+

Again, this is a recursive method. It checks four different possibilities.

+
    +
  1. First, we check if the query segment is equal to the segment for which our current node is responsible. If it is we just return this node's value.
    2. +
+

equalSegments

+
    +
  1. Does the query segment fully lie within the right child? If so, recursively perform the query on the right child.
    2. +
+

rightSegment

+
    +
  1. Does the query segment fully lie within the left child? If so, recursively perform the query on the left child.
    2. +
+

leftSegment

+
    +
  1. If none of the above, it means our query partially lies in both children so we combine the results of queries on both children.
    2. +
+

mixedSegment

+

This is how you can test it out in a playground:

+
let array = [1, 2, 3, 4]
+
+let sumSegmentTree = SegmentTree(array: array, function: +)
+
+sumSegmentTree.query(withLeftBound: 0, rightBound: 3)  // 1 + 2 + 3 + 4 = 10
+sumSegmentTree.query(withLeftBound: 1, rightBound: 2)  // 2 + 3 = 5
+sumSegmentTree.query(withLeftBound: 0, rightBound: 0)  // just 1
+sumSegmentTree.query(withLeftBound: 3, rightBound: 3)  // just 4
+

Querying the tree takes O(log n) time.

+

Replacing items

+

The value of a node in the segment tree depends on the nodes below it. So if we want to change a value of a leaf node, we need to update all its parent nodes too.

+

Here is the code:

+
  public func replaceItem(at index: Int, withItem item: T) {
+    if leftBound == rightBound {
+      value = item
+    } else if let leftChild = leftChild, rightChild = rightChild {
+      if leftChild.rightBound >= index {
+        leftChild.replaceItem(at: index, withItem: item)
+      } else {
+        rightChild.replaceItem(at: index, withItem: item)
+      }
+      value = function(leftChild.value, rightChild.value)
+    }
+  }
+

As usual, this works with recursion. If the node is a leaf, we just change its value. If the node is not a leaf, then we recursively call replaceItem(at: ) to update its children. After that, we recalculate the node's own value so that it is up-to-date again.

+

Replacing an item takes O(log n) time.

+

See the playground for more examples of how to use the segment tree.

+

See also

+

Lazy Propagation implementation and explanation.

+

Segment tree at PEGWiki

+

Written for Swift Algorithm Club by Artur Antonov

+ + diff --git a/Select Minimum Maximum/index.html b/Select Minimum Maximum/index.html new file mode 100644 index 000000000..86df7d05a --- /dev/null +++ b/Select Minimum Maximum/index.html @@ -0,0 +1,103 @@ + + + Select Minimum Maximum + + + +

Select Minimum / Maximum

+

Goal: Find the minimum/maximum object in an unsorted array.

+

Maximum or minimum

+

We have an array of generic objects and we iterate over all the objects keeping track of the minimum/maximum element so far.

+

An example

+

Let's say the we want to find the maximum value in the unsorted list [ 8, 3, 9, 4, 6 ].

+

Pick the first number, 8, and store it as the maximum element so far.

+

Pick the next number from the list, 3, and compare it to the current maximum. 3 is less than 8 so the maximum 8 does not change.

+

Pick the next number from the list, 9, and compare it to the current maximum. 9 is greater than 8 so we store 9 as the maximum.

+

Repeat this process until the all elements in the list have been processed.

+

The code

+

Here is a simple implementation in Swift:

+
func minimum<T: Comparable>(_ array: [T]) -> T? {
+  guard var minimum = array.first else {
+    return nil
+  }
+
+  for element in array.dropFirst() {
+    minimum = element < minimum ? element : minimum
+  }
+  return minimum
+}
+
+func maximum<T: Comparable>(_ array: [T]) -> T? {
+  guard var maximum = array.first else {
+    return nil
+  }
+
+  for element in array.dropFirst() {
+    maximum = element > maximum ? element : maximum
+  }
+  return maximum
+}
+

Put this code in a playground and test it like so:

+
let array = [ 8, 3, 9, 4, 6 ]
+minimum(array)   // This will return 3
+maximum(array)   // This will return 9
+

In the Swift standard library

+

The Swift library already contains an extension to SequenceType that returns the minimum/maximum element in a sequence.

+
let array = [ 8, 3, 9, 4, 6 ]
+array.minElement()   // This will return 3
+array.maxElement()   // This will return 9
+
let array = [ 8, 3, 9, 4, 6 ]
+//swift3
+array.min()   // This will return 3
+array.max()   // This will return 9
+

Maximum and minimum

+

To find both the maximum and minimum values contained in array while minimizing the number of comparisons we can compare the items in pairs.

+

An example

+

Let's say the we want to find the minimum and maximum value in the unsorted list [ 8, 3, 9, 6, 4 ].

+

Pick the first number, 8, and store it as the minimum and maximum element so far.

+

Because we have an odd number of items we remove 8 from the list which leaves the pairs [ 3, 9 ] and [ 6, 4 ].

+

Pick the next pair of numbers from the list, [ 3, 9 ]. Of these two numbers, 3 is the smaller one, so we compare 3 to the current minimum 8, and we compare 9 to the current maximum 8. 3 is less than 8 so the new minimum is 3. 9 is greater than 8 so the new maximum is 9.

+

Pick the next pair of numbers from the list, [ 6, 4 ]. Here, 4 is the smaller one, so we compare 4 to the current minimum 3, and we compare 6 to the current maximum 9. 4 is greater than 3 so the minimum does not change. 6 is less than 9 so the maximum does not change.

+

The result is a minimum of 3 and a maximum of 9.

+

The code

+

Here is a simple implementation in Swift:

+
func minimumMaximum<T: Comparable>(_ array: [T]) -> (minimum: T, maximum: T)? {
+  guard var minimum = array.first else {
+    return nil
+  }
+  var maximum = minimum
+
+  // if 'array' has an odd number of items, let 'minimum' or 'maximum' deal with the leftover
+  let start = array.count % 2 // 1 if odd, skipping the first element
+  for i in stride(from: start, to: array.count, by: 2) {
+    let pair = (array[i], array[i+1])
+
+    if pair.0 > pair.1 {
+      if pair.0 > maximum {
+        maximum = pair.0
+      }
+      if pair.1 < minimum {
+        minimum = pair.1
+      }
+    } else {
+      if pair.1 > maximum {
+        maximum = pair.1
+      }
+      if pair.0 < minimum {
+        minimum = pair.0
+      }
+    }
+  }
+
+  return (minimum, maximum)
+}
+

Put this code in a playground and test it like so:

+
let result = minimumMaximum(array)!
+result.minimum   // This will return 3
+result.maximum   // This will return 9
+

By picking elements in pairs and comparing their maximum and minimum with the running minimum and maximum we reduce the number of comparisons to 3 for every 2 elements.

+

Performance

+

These algorithms run at O(n). Each object in the array is compared with the running minimum/maximum so the time it takes is proportional to the array length.

+

Written by Chris Pilcher

+ + diff --git a/Selection Sampling/index.html b/Selection Sampling/index.html new file mode 100644 index 000000000..7d6902fbf --- /dev/null +++ b/Selection Sampling/index.html @@ -0,0 +1,164 @@ + + + Selection Sampling + + + +

Selection Sampling

+

Goal: Select k items at random from a collection of n items.

+

Let's say you have a deck of 52 playing cards and you need to draw 10 cards at random. This algorithm lets you do that.

+

Here's a very fast version:

+
func select<T>(from a: [T], count k: Int) -> [T] {
+  var a = a
+  for i in 0..<k {
+    let r = random(min: i, max: a.count - 1)
+    if i != r {
+      swap(&a[i], &a[r])
+    }
+  }
+  return Array(a[0..<k])
+}
+

As often happens with these kinds of algorithms, it divides the array into two regions. The first region contains the selected items; the second region is all the remaining items.

+

Here's an example. Let's say the array is:

+
[ "a", "b", "c", "d", "e", "f", "g" ]
+
+

We want to select 3 items, so k = 3. In the loop, i is initially 0, so it points at "a".

+
[ "a", "b", "c", "d", "e", "f", "g" ]
+   i
+
+

We calculate a random number between i and a.count, the size of the array. Let's say this is 4. Now we swap "a" with "e", the element at index 4, and move i forward:

+
[ "e" | "b", "c", "d", "a", "f", "g" ]
+         i
+
+

The | bar shows the split between the two regions. "e" is the first element we've selected. Everything to the right of the bar we still need to look at.

+

Again, we ask for a random number between i and a.count, but because i has shifted, the random number can never be less than 1. So we'll never again swap "e" with anything.

+

Let's say the random number is 6 and we swap "b" with "g":

+
[ "e" , "g" | "c", "d", "a", "f", "b" ]
+               i
+
+

One more random number to pick, let's say it is 4 again. We swap "c" with "a" to get the final selection on the left:

+
[ "e", "g", "a" | "d", "c", "f", "b" ]
+
+

And that's it. Easy peasy. The performance of this function is O(k) because as soon as we've selected k elements, we're done.

+

Here is an alternative algorithm, called "reservoir sampling":

+
func reservoirSample<T>(from a: [T], count k: Int) -> [T] {
+  precondition(a.count >= k)
+
+  var result = [T]()      // 1
+  for i in 0..<k {
+    result.append(a[i])
+  }
+
+  for i in k..<a.count {  // 2
+    let j = random(min: 0, max: i)
+    if j < k {
+      result[j] = a[i]
+    }
+  }
+  return result
+}
+

This works in two steps:

+
    +
  1. Fill the result array with the first k elements from the original array. This is called the "reservoir".
    2. +
  2. Randomly replace elements in the reservoir with elements from the remaining pool.
    3. +
+

The performance of this algorithm is O(n), so it's a little bit slower than the first algorithm. However, its big advantage is that it can be used for arrays that are too large to fit in memory, even if you don't know what the size of the array is (in Swift this might be something like a lazy generator that reads the elements from a file).

+

There is one downside to the previous two algorithms: they do not keep the elements in the original order. In the input array "a" came before "e" but now it's the other way around. If that is an issue for your app, you can't use this particular method.

+

Here is an alternative approach that does keep the original order intact, but is a little more involved:

+
func select<T>(from a: [T], count requested: Int) -> [T] {
+  var examined = 0
+  var selected = 0
+  var b = [T]()
+  
+  while selected < requested {                          // 1
+    let r = Double(arc4random()) / 0x100000000          // 2
+    
+    let leftToExamine = a.count - examined              // 3
+    let leftToAdd = requested - selected
+
+    if Double(leftToExamine) * r < Double(leftToAdd) {  // 4
+      selected += 1
+      b.append(a[examined])
+    }
+
+    examined += 1
+  }
+  return b
+}
+

This algorithm uses probability to decide whether to include a number in the selection or not.

+
    +
  1. +

    The loop steps through the array from beginning to end. It keeps going until we've selected k items from our set of n. Here, k is called requested and n is a.count.

    +
    2. +
  2. +

    Calculate a random number between 0 and 1. We want 0.0 <= r < 1.0. The higher bound is exclusive; we never want it to be exactly 1. That's why we divide the result from arc4random() by 0x100000000 instead of the more usual 0xffffffff.

    +
    3. +
  3. +

    leftToExamine is how many items we still haven't looked at. leftToAdd is how many items we still need to select before we're done.

    +
    4. +
  4. +

    This is where the magic happens. Basically, we're flipping a coin. If it was heads, we add the current array element to the selection; if it was tails, we skip it.

    +
    5. +
+

Interestingly enough, even though we use probability, this approach always guarantees that we end up with exactly k items in the output array.

+

Let's walk through the same example again. The input array is:

+
[ "a", "b", "c", "d", "e", "f", "g" ]
+
+

The loop looks at each element in turn, so we start at "a". We get a random number between 0 and 1, let's say it is 0.841. The formula at // 4 multiplies the number of items left to examine with this random number. There are still 7 elements left to examine, so the result is:

+
7 * 0.841 = 5.887
+
+

We compare this to 3 because we wanted to select 3 items. Since 5.887 is greater than 3, we skip "a" and move on to "b".

+

Again, we get a random number, let's say 0.212. Now there are only 6 elements left to examine, so the formula gives:

+
6 * 0.212 = 1.272
+
+

This is less than 3 and we add "b" to the selection. This is the first item we've selected, so two left to go.

+

On to the next element, "c". The random number is 0.264, giving the result:

+
5 * 0.264 = 1.32
+
+

There are only 2 elements left to select, so this number must be less than 2. It is, and we also add "c" to the selection. The total selection is [ "b", "c" ].

+

Only one item left to select but there are still 4 candidates to look at. Suppose the next random number is 0.718. The formula now gives:

+
4 * 0.718 = 2.872
+
+

For this element to be selected the number has to be less than 1, as there is only 1 element left to be picked. It isn't, so we skip "d". Only three possibilities left -- will we make it before we run out of elements?

+

The random number is 0.346. The formula gives:

+
3 * 0.346 = 1.038
+
+

Just a tiny bit too high. We skip "e". Only two candidates left...

+

Note that now literally we're dealing with a coin toss: if the random number is less than 0.5 we select "f" and we're done. If it's greater than 0.5, we go on to the final element. Let's say we get 0.583:

+
2 * 0.583 = 1.166
+
+

We skip "f" and look at the very last element. Whatever random number we get here, it should always select "g" or we won't have selected enough elements and the algorithm doesn't work!

+

Let's say our final random number is 0.999 (remember, it can never be 1.0 or higher). Actually, no matter what we choose here, the formula will always give a value less than 1:

+
1 * 0.999 = 0.999
+
+

And so the last element will always be chosen if we didn't have a big enough selection yet. The final selection is [ "b", "c", "g" ]. Notice that the elements are still in their original order, because we examined the array from left to right.

+

Maybe you're not convinced yet... What if we always got 0.999 as the random value (the maximum possible), would that still select 3 items? Well, let's do the math:

+
7 * 0.999 = 6.993     is this less than 3? no
+6 * 0.999 = 5.994     is this less than 3? no
+5 * 0.999 = 4.995     is this less than 3? no
+4 * 0.999 = 3.996     is this less than 3? no
+3 * 0.999 = 2.997     is this less than 3? YES
+2 * 0.999 = 1.998     is this less than 2? YES
+1 * 0.999 = 0.999     is this less than 1? YES
+
+

It always works! But does this mean that elements closer to the end of the array have a higher probability of being chosen than those in the beginning? Nope, all elements are equally likely to be selected. (Don't take my word for it: see the playground for a quick test that shows this in practice.)

+

Here's an example of how to test this algorithm:

+
let input = [
+  "there", "once", "was", "a", "man", "from", "nantucket",
+  "who", "kept", "all", "of", "his", "cash", "in", "a", "bucket",
+  "his", "daughter", "named", "nan",
+  "ran", "off", "with", "a", "man",
+  "and", "as", "for", "the", "bucket", "nan", "took", "it",
+]
+
+let output = select(from: input, count: 10)
+print(output)
+print(output.count)
+

The performance of this second algorithm is O(n) as it may require a pass through the entire input array.

+
+

Note: If k > n/2, then it's more efficient to do it the other way around and choose a.count - k items to remove.

+
+

Based on code from Algorithm Alley, Dr. Dobb's Magazine, October 1993.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Selection Sort/index.html b/Selection Sort/index.html new file mode 100644 index 000000000..ee642a06d --- /dev/null +++ b/Selection Sort/index.html @@ -0,0 +1,103 @@ + + + Selection Sort + + + +

Selection Sort

+

Goal: To sort an array from low to high (or high to low).

+

You are given an array of numbers and need to put them in the right order. The selection sort algorithm divides the array into two parts: the beginning of the array is sorted, while the rest of the array consists of the numbers that still remain to be sorted.

+
[ ...sorted numbers... | ...unsorted numbers... ]
+
+

This is similar to insertion sort, but the difference is in how new numbers are added to the sorted portion.

+

It works as follows:

+ +

It is called a "selection" sort because at every step you search through the rest of the array to select the next lowest number.

+

An example

+

Suppose the numbers to sort are [ 5, 8, 3, 4, 6 ]. We also keep track of where the sorted portion of the array ends, denoted by the | symbol.

+

Initially, the sorted portion is empty:

+
[| 5, 8, 3, 4, 6 ]
+
+

Now we find the lowest number in the array. We do that by scanning through the array from left to right, starting at the | bar. We find the number 3.

+

To put this number into the sorted position, we swap it with the number next to the |, which is 5:

+
[ 3 | 8, 5, 4, 6 ]
+  *      *
+
+

The sorted portion is now [ 3 ] and the rest is [ 8, 5, 4, 6 ].

+

Again, we look for the lowest number, starting from the | bar. We find 4 and swap it with 8 to get:

+
[ 3, 4 | 5, 8, 6 ]
+     *      *
+
+

With every step, the | bar moves one position to the right. We again look through the rest of the array and find 5 as the lowest number. There is no need to swap 5 with itself, and we simply move forward:

+
[ 3, 4, 5 | 8, 6 ]
+        *
+
+

This process repeats until the array is sorted. Note that everything to the left of the | bar is always in sorted order and always contains the lowest numbers in the array. Finally, we end up with:

+
[ 3, 4, 5, 6, 8 |]
+
+

The selection sort is an in-place sort because everything happens in the same array without using additional memory. You can also implement this as a stable sort so that identical elements do not get swapped around relative to each other (note that the version given below is not stable).

+

The code

+

Here is an implementation of selection sort in Swift:

+
func selectionSort(_ array: [Int]) -> [Int] {
+  guard array.count > 1 else { return array }  // 1
+
+  var a = array                    // 2
+
+  for x in 0 ..< a.count - 1 {     // 3
+
+    var lowest = x
+    for y in x + 1 ..< a.count {   // 4
+      if a[y] < a[lowest] {
+        lowest = y
+      }
+    }
+
+    if x != lowest {               // 5
+      a.swapAt(x, lowest)
+    }
+  }
+  return a
+}
+

Put this code in a playground and test it like so:

+
let list = [ 10, -1, 3, 9, 2, 27, 8, 5, 1, 3, 0, 26 ]
+selectionSort(list)
+

A step-by-step explanation of how the code works:

+
    +
  1. +

    If the array is empty or only contains a single element, then there is no need to sort.

    +
    2. +
  2. +

    Make a copy of the array. This is necessary because we cannot modify the contents of the array parameter directly in Swift. Like the Swift's sort() function, the selectionSort() function will return a sorted copy of the original array.

    +
    3. +
  3. +

    There are two loops inside this function. The outer loop looks at each of the elements in the array in turn; this is what moves the | bar forward.

    +
    4. +
  4. +

    This is the inner loop. It finds the lowest number in the rest of the array.

    +
    5. +
  5. +

    Swap the lowest number with the current array index. The if check is necessary because you can't swap() an element with itself in Swift.

    +
    6. +
+

In summary: For each element of the array, the selection sort swaps positions with the lowest value from the rest of the array. As a result, the array gets sorted from the left to the right. (You can also do it right-to-left, in which case you always look for the largest number in the array. Give that a try!)

+
+

Note: The outer loop ends at index a.count - 2. The very last element will automatically be in the correct position because at that point there are no other smaller elements left.

+
+

The source file SelectionSort.swift has a version of this function that uses generics, so you can also use it to sort strings and other data types.

+

Performance

+

The selection sort is easy to understand but it performs slow as O(n^2). It is worse than insertion sort but better than bubble sort. Finding the lowest element in the rest of the array is slow, especially since the inner loop will be performed repeatedly.

+

The Heap sort uses the same principle as selection sort but has a fast method for finding the minimum value in the rest of the array. The heap sort' performance is O(n log n).

+

See also

+

Selection sort on Wikipedia

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Set Cover (Unweighted)/index.html b/Set Cover (Unweighted)/index.html new file mode 100644 index 000000000..3aa4c8fd2 --- /dev/null +++ b/Set Cover (Unweighted)/index.html @@ -0,0 +1,30 @@ + + + Set Cover (Unweighted) + + + +

Set Cover (Unweighted)

+

If you have a group of sets, this algorithm finds a subset of those sets within that group whose union will cover an initial set that you're trying to match. The initial set is also known as the universe.

+

For example, suppose you have a universe of {1, 5, 7} and you want to find the sets which cover the universe within the following group of sets:

+
+

{8, 4, 2}
+{3, 1}
+{7, 6, 5, 4}
+{2}
+{1, 2, 3}

+
+

You can see that the sets {3, 1} {7, 6, 5, 4} when unioned together will cover the universe of {1, 5, 7}. Yes, there may be additional elements in the sets returned by the algorithm, but every element in the universe is represented in the cover itself.

+

There may be cases where no cover exists. For example, if your universe is {7, 9}, there is no combination of sets within the group above that will yield a cover.

+

The algorithm

+

The Greedy Set Cover algorithm (unweighted) is provided here. It's known as greedy because it uses the largest intersecting set from the group of sets first before examining other sets in the group. This is part of the reason why the cover may have additional elements which are not part of the universe.

+

The function (named cover) is provided as an extension of the Swift type Set. The function takes a single parameter, which is an array of sets. This array represents the group, and the set itself represents the universe.

+

One of the first things done in cover is to make a copy of the universe in remainingSet. Then, the algorithm enters a while loop in which a call to largestIntersectingSet is made. The value returned from largestIntersectingSet is the set which has the most elements in common with the remaining universe identified by remainingSet. If all sets have nothing in common, largestIntersectingSet returns nil.

+

If the result from largestIntersectingSet is not nil, that result is subtracted from remainingSet (reducing its size), and the loop continues until remainingSet has zero length (meaning a cover has been found) or until largestIntersectingSet returns nil.

+

If there is no cover within the group of sets, cover returns nil.

+

See also

+

Set cover problem on Wikipedia

+

Written for Swift Algorithm Club by Michael C. Rael
+Migrated to Swift 3 by Jaap Wijnen

+ + diff --git a/Shell Sort/index.html b/Shell Sort/index.html new file mode 100644 index 000000000..1d04de090 --- /dev/null +++ b/Shell Sort/index.html @@ -0,0 +1,106 @@ + + + Shell Sort + + + +

Shell Sort

+

Shell sort is based on insertion sort as a general way to improve its performance, by breaking the original list into smaller sublists which are then individually sorted using insertion sort.

+

There is a nice video created at Sapientia University which shows the process as a Hungarian folk dance.

+

How it works

+

Instead of comparing elements that are side-by-side and swapping them if they are out of order, the way insertion sort does it, the shell sort algorithm compares elements that are far apart.

+

The distance between elements is known as the gap. If the elements being compared are in the wrong order, they are swapped across the gap. This eliminates many in-between copies that are common with insertion sort.

+

The idea is that by moving the elements over large gaps, the array becomes partially sorted quite quickly. This makes later passes faster because they don't have to swap so many items anymore.

+

Once a pass has been completed, the gap is made smaller and a new pass starts. This repeats until the gap has size 1, at which point the algorithm functions just like insertion sort. But since the data is already fairly well sorted by then, the final pass can be very quick.

+

An example

+

Suppose we want to sort the array [64, 20, 50, 33, 72, 10, 23, -1, 4] using shell sort.

+

We start by dividing the length of the array by 2:

+
n = floor(9/2) = 4
+
+

This is the gap size.

+

We create n sublists. In each sublist, the items are spaced apart by a gap of size n. In our example, we need to make four of these sublists. The sublists are sorted by the insertionSort() function.

+

That may not have made a whole lot of sense, so let's take a closer look at what happens.

+

The first pass is as follows. We have n = 4, so we make four sublists:

+
sublist 0:  [ 64, xx, xx, xx, 72, xx, xx, xx, 4  ]
+sublist 1:  [ xx, 20, xx, xx, xx, 10, xx, xx, xx ]
+sublist 2:  [ xx, xx, 50, xx, xx, xx, 23, xx, xx ]
+sublist 3:  [ xx, xx, xx, 33, xx, xx, xx, -1, xx ]
+
+

As you can see, each sublist contains only every 4th item from the original array. The items that are not in a sublist are marked with xx. So the first sublist is [ 64, 72, 4 ] and the second is [ 20, 10 ], and so on. The reason we use this "gap" is so that we don't have to actually make new arrays. Instead, we interleave them in the original array.

+

We now call insertionSort() once on each sublist.

+

This particular version of insertion sort sorts from the back to the front. Each item in the sublist is compared against the others. If they're in the wrong order, the value is swapped and travels all the way down until we reach the start of the sublist.

+

So for sublist 0, we swap 4 with 72, then swap 4 with 64. After sorting, this sublist looks like:

+
sublist 0:  [ 4, xx, xx, xx, 64, xx, xx, xx, 72 ]
+
+

The other three sublists after sorting:

+
sublist 1:  [ xx, 10, xx, xx, xx, 20, xx, xx, xx ]
+sublist 2:  [ xx, xx, 23, xx, xx, xx, 50, xx, xx ]
+sublist 3:  [ xx, xx, xx, -1, xx, xx, xx, 33, xx ]
+
+

The total array looks like this now:

+
[ 4, 10, 23, -1, 64, 20, 50, 33, 72 ]
+
+

It's not entirely sorted yet but it's more sorted than before. This completes the first pass.

+

In the second pass, we divide the gap size by two:

+
n = floor(4/2) = 2
+
+

That means we now create only two sublists:

+
sublist 0:  [  4, xx, 23, xx, 64, xx, 50, xx, 72 ]
+sublist 1:  [ xx, 10, xx, -1, xx, 20, xx, 33, xx ]
+
+

Each sublist contains every 2nd item. Again, we call insertionSort() to sort these sublists. The result is:

+
sublist 0:  [  4, xx, 23, xx, 50, xx, 64, xx, 72 ]
+sublist 1:  [ xx, -1, xx, 10, xx, 20, xx, 33, xx ]
+
+

Note that in each list only two elements were out of place. So the insertion sort is really fast. That's because we already sorted the array a little in the first pass.

+

The total array looks like this now:

+
[ 4, -1, 23, 10, 50, 20, 64, 33, 72 ]
+
+

This completes the second pass. The gap size of the final pass is:

+
n = floor(2/2) = 1
+
+

A gap size of 1 means we only have a single sublist, the array itself, and once again we call insertionSort() to sort it. The final sorted array is:

+
[ -1, 4, 10, 20, 23, 33, 50, 64, 72 ]
+
+

The performance of shell sort is O(n^2) in most cases or O(n log n) if you get lucky. This algorithm produces an unstable sort; it may change the relative order of elements with equal values.

+

The gap sequence

+

The "gap sequence" determines the initial size of the gap and how it is made smaller with each iteration. A good gap sequence is important for shell sort to perform well.

+

The gap sequence in this implementation is the one from Shell's original version: the initial value is half the array size and then it is divided by 2 each time. There are other ways to calculate the gap sequence.

+

Just for fun...

+

This is an old Commodore 64 BASIC version of shell sort that Matthijs used a long time ago and ported to pretty much every programming language he ever used:

+
61200 REM S is the array to be sorted
+61205 REM AS is the number of elements in S
+61210 W1=AS
+61220 IF W1<=0 THEN 61310
+61230 W1=INT(W1/2): W2=AS-W1
+61240 V=0
+61250 FOR N1=0 TO W2-1
+61260 W3=N1+W1
+61270 IF S(N1)>S(W3) THEN SH=S(N1): S(N1)=S(W3): S(W3)=SH: V=1
+61280 NEXT N1
+61290 IF V>0 THEN 61240
+61300 GOTO 61220
+61310 RETURN
+
+

The Code:

+

Here is an implementation of Shell Sort in Swift:

+
var arr = [64, 20, 50, 33, 72, 10, 23, -1, 4, 5]
+
+public func shellSort(_ list: inout [Int]) {
+    var sublistCount = list.count / 2
+    while sublistCount > 0 {
+        for pos in 0..<sublistCount {
+            insertionSort(&list, start: pos, gap: sublistCount)
+        }
+        sublistCount = sublistCount / 2
+    }
+}
+
+shellSort(&arr)
+
+

See also

+

Shellsort on Wikipedia

+

Shell sort at Rosetta code

+

Written for Swift Algorithm Club by Mike Taghavi and Matthijs Hollemans

+ + diff --git a/Shortest Path (Unweighted)/index.html b/Shortest Path (Unweighted)/index.html new file mode 100644 index 000000000..a7fac60c9 --- /dev/null +++ b/Shortest Path (Unweighted)/index.html @@ -0,0 +1,74 @@ + + + Shortest Path (Unweighted) + + + +

Shortest Path (Unweighted Graph)

+

Goal: find the shortest route to go from one node to another in a graph.

+

Suppose we have to following graph:

+

Example graph

+

We may want to find out what the shortest way is to get from node A to node F.

+

If the graph is unweighed, then finding the shortest path is easy: we can use the breadth-first search algorithm. For a weighted graph, we can use Dijkstra's algorithm.

+

Unweighted graph: breadth-first search

+

Breadth-first search is a method for traversing a tree or graph data structure. It starts at a source node and explores the immediate neighbor nodes first, before moving to the next level neighbors. As a convenient side effect, it automatically computes the shortest path between a source node and each of the other nodes in the tree or graph.

+

The result of the breadth-first search can be represented with a tree:

+

The BFS tree

+

The root of the tree is the node you started the breadth-first search from. To find the distance from node A to any other node, we simply count the number of edges in the tree. And so we find that the shortest path between A and F is 2. The tree not only tells you how long that path is, but also how to actually get from A to F (or any of the other nodes).

+

Let's put breadth-first search into practice and calculate the shortest path from A to all the other nodes. We start with the source node A and add it to a queue with a distance of 0.

+
queue.enqueue(element: A)
+A.distance = 0
+

The queue is now [ A ]. We dequeue A and enqueue its two immediate neighbor nodes B and C with a distance of 1.

+
queue.dequeue()   // A
+queue.enqueue(element: B)
+B.distance = A.distance + 1   // result: 1
+queue.enqueue(element: C)
+C.distance = A.distance + 1   // result: 1
+

The queue is now [ B, C ]. Dequeue B and enqueue B's neighbor nodes D and E with a distance of 2.

+
queue.dequeue()   // B
+queue.enqueue(element: D)
+D.distance = B.distance + 1   // result: 2
+queue.enqueue(element: E)
+E.distance = B.distance + 1   // result: 2
+

The queue is now [ C, D, E ]. Dequeue C and enqueue C's neighbor nodes F and G, also with a distance of 2.

+
queue.dequeue()   // C
+queue.enqueue(element: F)
+F.distance = C.distance + 1   // result: 2
+queue.enqueue(element: G)
+G.distance = C.distance + 1   // result: 2
+

This continues until the queue is empty and we've visited all the nodes. Each time we discover a new node, it gets the distance of its parent plus 1. As you can see, this is exactly what the breadth-first search algorithm does, except that we now also keep track of the distance travelled.

+

Here's the code:

+
func breadthFirstSearchShortestPath(graph: Graph, source: Node) -> Graph {
+  let shortestPathGraph = graph.duplicate()
+
+  var queue = Queue<Node>()
+  let sourceInShortestPathsGraph = shortestPathGraph.findNodeWithLabel(label: source.label)
+  queue.enqueue(element: sourceInShortestPathsGraph)
+  sourceInShortestPathsGraph.distance = 0
+
+  while let current = queue.dequeue() {
+    for edge in current.neighbors {
+      let neighborNode = edge.neighbor
+      if !neighborNode.hasDistance {
+        queue.enqueue(element: neighborNode)
+        neighborNode.distance = current.distance! + 1
+      }
+    }
+  }
+
+  return shortestPathGraph
+}
+

Put this code in a playground and test it like so:

+
let shortestPathGraph = breadthFirstSearchShortestPath(graph: graph, source: nodeA)
+print(shortestPathGraph.nodes)
+

This will output:

+
Node(label: a, distance: 0), Node(label: b, distance: 1), Node(label: c, distance: 1),
+Node(label: d, distance: 2), Node(label: e, distance: 2), Node(label: f, distance: 2),
+Node(label: g, distance: 2), Node(label: h, distance: 3)
+
+
+

Note: This version of breadthFirstSearchShortestPath() does not actually produce the tree, it only computes the distances. See minimum spanning tree on how you can convert the graph into a tree by removing edges.

+
+

Written by Chris Pilcher and Matthijs Hollemans

+ + diff --git a/Shuffle/index.html b/Shuffle/index.html new file mode 100644 index 000000000..acb8c8b6c --- /dev/null +++ b/Shuffle/index.html @@ -0,0 +1,89 @@ + + + Shuffle + + + +

Shuffle

+

Goal: Rearrange the contents of an array.

+

Imagine you're making a card game and you need to shuffle a deck of cards. You can represent the deck by an array of Card objects and shuffling the deck means to change the order of those objects in the array. (It's like the opposite of sorting.)

+

Here is a naive way to approach this in Swift:

+
extension Array {
+  public mutating func shuffle() {
+    var temp = [Element]()
+    while !isEmpty {
+      let i = random(count)
+      let obj = remove(at: i)
+      temp.append(obj)
+    }
+    self = temp
+  }
+}
+

To try it out, copy the code into a playground and then do:

+
var list = [ "a", "b", "c", "d", "e", "f", "g" ]
+list.shuffle()
+list.shuffle()
+list.shuffle()
+

You should see three different arrangements -- or permutations to use math-speak -- of the objects in the array.

+

This shuffle works in place, it modifies the contents of the original array. The algorithm works by creating a new array, temp, that is initially empty. Then we randomly choose an element from the original array and append it to temp, until the original array is empty. Finally, the temporary array is copied back into the original one.

+

This code works just fine but it's not very efficient. Removing an element from an array is an O(n) operation and we perform this n times, making the total algorithm O(n^2). We can do better!

+

The Fisher-Yates / Knuth shuffle

+

Here is a much improved version of the shuffle algorithm:

+
extension Array {
+  public mutating func shuffle() {
+    for i in stride(from: count - 1, through: 1, by: -1) {
+      let j = Int.random(in: 0...i)
+      if i != j {
+        swap(&self[i], &self[j])
+      }
+    }
+  }
+}
+

Again, this picks objects at random. In the naive version we placed those objects into a new temporary array so we could keep track of which objects were already shuffled and which still remained to be done. In this improved algorithm, however, we'll move the shuffled objects to the end of the original array.

+

Let's walk through the example. We have the array:

+
[ "a", "b", "c", "d", "e", "f", "g" ]
+
+

The loop starts at the end of the array and works its way back to the beginning. The very first random number can be any element from the entire array. Let's say it returns 2, the index of "c". We swap "c" with "g" to move it to the end:

+
[ "a", "b", "g", "d", "e", "f" | "c" ]
+             *                    *
+
+

The array now consists of two regions, indicated by the | bar. Everything to the right of the bar is shuffled already.

+

The next random number is chosen from the range 0...6, so only from the region [ "a", "b", "g", "d", "e", "f" ]. It will never choose "c" since that object is done and we'll no longer touch it.

+

Let's say the random number generator picks 0, the index of "a". Then we swap "a" with "f", which is the last element in the unshuffled portion, and the array looks like this:

+
[ "f", "b", "g", "d", "e" | "a", "c" ]
+   *                         *
+
+

The next random number is somewhere in [ "f", "b", "g", "d", "e" ], so let's say it is 3. We swap "d" with "e":

+
[ "f", "b", "g", "e" | "d", "a", "c" ]
+                  *     *
+
+

And so on... This continues until there is only one element remaining in the left portion. For example:

+
[ "b" | "e", "f", "g", "d", "a", "c" ]
+
+

There's nothing left to swap that "b" with, so we're done.

+

Because we only look at each array element once, this algorithm has a guaranteed running time of O(n). It's as fast as you could hope to get!

+

Creating a new array that is shuffled

+

There is a slight variation on this algorithm that is useful for when you want to create a new array instance that contains the values 0 to n-1 in random order.

+

Here is the code:

+
public func shuffledArray(_ n: Int) -> [Int] {
+  var a = [Int](repeating: 0, count: n)
+  for i in 0..<n {
+    let j = Int.random(in: 0...i)
+    // for the Fisher–Yates_shuffle's pseudo code implement in wiki, it will check if i != j
+    a[i] = a[j]
+    a[j] = i
+  }
+  return a
+}
+

To use it:

+
let numbers = shuffledArray(10)
+

This returns something like [3, 0, 9, 1, 8, 5, 2, 6, 7, 4]. As you can see, every number between 0 and 10 is in that list, but shuffled around. Of course, when you try it for yourself the order of the numbers will be different.

+

The shuffledArray() function first creates a new array with n zeros. Then it loops n times and in each step adds the next number from the sequence to a random position in the array. The trick is to make sure that none of these numbers gets overwritten with the next one, so it moves the previous number out of the way first!

+

For this function, The condition that checks if j ≠ i may be omitted in languages that have no problems accessing uninitialized array values, and for which assigning is cheaper than comparing., you can check it in wiki. And also remove checking logic will optimise performance.

+

The algoritm is quite clever and I suggest you walk through an example yourself, either on paper or in the playground. (Hint: Again it splits the array into two regions.)

+

See also

+

These Swift implementations are based on pseudocode from the Wikipedia article.

+

Mike Bostock has a great visualization of the shuffle algorithm.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Shunting Yard/index.html b/Shunting Yard/index.html new file mode 100644 index 000000000..5c2ca9f74 --- /dev/null +++ b/Shunting Yard/index.html @@ -0,0 +1,231 @@ + + + Shunting Yard + + + +

Shunting Yard Algorithm

+

Any mathematics we write is expressed in a notation known as infix notation. For example:

+

A + B * C

+

The operator is placed in between the operands, hence the expression is said to be in infix form. If you think about it, any expression that you write on a piece of paper will always be in infix form. This is what we humans understand.

+

Multiplication has higher precedence than addition, so when the above expression is evaluated you would first multiply B and C, and then add the result to A. We humans can easily understand the precedence of operators, but a machine needs to be given instructions about each operator.

+

To change the order of evaluation, we can use parentheses:

+

(A + B) * C

+

Now A is first added to B and then the sum is multiplied by C.

+

If you were to write an algorithm that parsed and evaluated expressions in infix notation you will realize that it's a tedious process. You'd have to parse the expression multiple times to know what operation to perform first. As the number of operators increase so does the complexity.

+

Postfix notation

+

In postfix notation, also known as Reverse Polish Notation or RPN, the operators come after the corresponding operands. Here is the postfix representation of the example from the previous section:

+

A B C * +

+

An expression in postfix form will not have any parentheses and neither will you have to worry about scanning for operator precedence. This makes it easy for the computer to evaluate expressions, since the order in which the operators need to be applied is fixed.

+

Evaluating a postfix expression is easily done using a stack. Here is the pseudocode:

+
    +
  1. Read the postfix expression token by token
    2. +
  2. If the token is an operand, push it onto the stack
    3. +
  3. If the token is a binary operator, +
      +
    1. Pop the two topmost operands from the stack
      2. +
    2. Apply the binary operator to the two operands
      3. +
    3. Push the result back onto the stack
      4. +
    +
    4. +
  4. Finally, the value of the whole postfix expression remains in the stack
    5. +
+

Using the above pseudocode, the evaluation of A B C * + would be as follows:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ExpressionStack
A B C * +
B C * +A
C * +A, B
* +A, B, C
+A, D
E
+

Where D = B * C and E = A + D.

+

As seen above, a postfix operation is relatively easy to evaluate as the order in which the operators need to be applied is pre-decided.

+

Converting infix to postfix

+

The shunting yard algorithm was invented by Edsger Dijkstra to convert an infix expression to postfix. Many calculators use this algorithm to convert the expression being entered to postfix form.

+

Here is the psedocode of the algorithm:

+
    +
  1. For all the input tokens: +
      +
    1. Read the next token
      2. +
    2. If token is an operator (x) +
        +
      1. While there is an operator (y) at the top of the operators stack and either (x) is left-associative and its precedence is less or equal to that of (y), or (x) is right-associative and its precedence is less than (y) +
          +
        1. Pop (y) from the stack
          2. +
        2. Add (y) output buffer
          3. +
        +
        2. +
      2. Push (x) on the stack
        3. +
      +
      3. +
    3. Else if token is left parenthesis, then push it on the stack
      4. +
    4. Else if token is a right parenthesis +
        +
      1. Until the top token (from the stack) is left parenthesis, pop from the stack to the output buffer
        2. +
      2. Also pop the left parenthesis but don’t include it in the output buffer
        3. +
      +
      5. +
    5. Else add token to output buffer
      6. +
    +
    2. +
  2. Pop any remaining operator tokens from the stack to the output
    3. +
+

Let's take a small example and see how the pseudocode works. Here is the infix expression to convert:

+

4 + 4 * 2 / ( 1 - 5 )

+

The following table describes the precedence and the associativity for each operator. The same values are used in the algorithm.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
OperatorPrecedenceAssociativity
^10Right
*5Left
/5Left
+0Left
-0Left
+

Here we go:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TokenActionOutputOperator stack
4Add token to output4
+Push token to stack4+
4Add token to output4 4+
*Push token to stack4 4* +
2Add token to output4 4 2* +
/Pop stack to output, Push token to stack4 4 2 */ +
(Push token to stack4 4 2 *( / +
1Add token to output4 4 2 * 1( / +
-Push token to stack4 4 2 * 1- ( / +
5Add token to output4 4 2 * 1 5- ( / +
)Pop stack to output, Pop stack4 4 2 * 1 5 -/ +
endPop entire stack to output4 4 2 * 1 5 - / +
+

We end up with the postfix expression:

+

4 4 2 * 1 5 - / +

+

See also

+

Shunting yard algorithm on Wikipedia

+

Written for the Swift Algorithm Club by Ali Hafizji
+Migrated to Swift 3 by Jaap Wijnen

+ + diff --git a/Simulated annealing/index.html b/Simulated annealing/index.html new file mode 100644 index 000000000..eda829b0a --- /dev/null +++ b/Simulated annealing/index.html @@ -0,0 +1,35 @@ + + + Simulated annealing + + + +

Simulated annealing

+

Simulated Annealing is a nature inspired global optimization technique and a metaheuristic to approximate global maxima in a (often discrete)large search space. The name comes from the process of annealing in metallurgy where a material is heated and cooled down under controlled conditions in order to improve its strength and durabilility. The objective is to find a minimum cost solution in the search space by exploiting properties of a thermodynamic system.
+Unlike hill climbing techniques which usually gets stuck in a local maxima ( downward moves are not allowed ), simulated annealing can escape local maxima. The interesting property of simulated annealing is that probability of allowing downward moves is high at the high temperatures and gradually reduced as it cools down. In other words, high temperature relaxes the acceptance criteria for the search space and triggers chaotic behavior of acceptance function in the algorithm (e.x initial/high temperature stages) which should make it possible to escape from local maxima and cooler temperatures narrows it and focuses on improvements.

+

Pseucocode

+
Input: initial, temperature, coolingRate, acceptance
+Output: Sbest
+Scurrent <- CreateInitialSolution(initial)
+Sbest <- Scurrent
+while temperature is not minimum:
+	Snew <- FindNewSolution(Scurrent)
+	if acceptance(Energy(Scurrent), Energy(Snew), temperature) > Rand():
+		Scurrent = Snew
+	if Energy(Scurrent) < Energy(Sbest):
+		Sbest = Scurrent
+	temperature = temperature * (1-coolingRate)
+
+

Common acceptance criteria :

+
P(accept) <- exp((e-ne)/T) where 
+	e is the current energy ( current solution ), 
+	ne is new energy ( new solution ),
+	T is current temperature.
+
+

We use this algorithm to solve a Travelling salesman problem instance with 20 cities. The code is in simann_example.swift

+

#See also

+

Simulated annealing on Wikipedia

+

Travelling salesman problem

+

Written for Swift Algorithm Club by Mike Taghavi

+ + diff --git a/Single-Source Shortest Paths (Weighted)/index.html b/Single-Source Shortest Paths (Weighted)/index.html new file mode 100644 index 000000000..455bee3d4 --- /dev/null +++ b/Single-Source Shortest Paths (Weighted)/index.html @@ -0,0 +1,228 @@ + + + Single-Source Shortest Paths (Weighted) + + + +

Single-Source Shortest Paths

+

The Single-Source shortest path problem finds the shortest paths from a given source vertex to all other vertices in a directed weighted graph. Many variations exist on the problem, specifying whether or not edges may have negative values, whether cycles exist, or whether a path between a specific pair of vertices.

+

Bellman-Ford

+

The Bellman-Ford shortest paths algorithm finds the shortest paths to all vertices from a given source vertex s in a directed graph that may contain negative edge weights. It iterates over all edges for each other vertex in the graph, applying a relaxation to the current state of known shortest path weights. The intuition here is that a path will not contain more vertices than are present in the graph, so performing a pass over all edges |V|-1 number of times is sufficient to compare all possible paths.

+

At each step, a value is stored for each vertex v, which is the weight of the current known shortest path s~>v. This value remains 0 for the source vertex itself, and all others are initially ∞. Then, they are "relaxed" by applying the following test to each edge e = (u, v) (where u is a source vertex and v is a destination vertex for the directed edge):

+
if weights[v] > weights[u] + e.weight {
+	weights[v] = weights[u] + e.weight
+}
+
+

Bellman-Ford in essence only computes the lengths of the shortest paths, but can optionally maintain a structure that memoizes the predecessor of each vertex on its shortest path from the source vertex. Then the paths themselves can be reconstructed by recursing through this structure from a destination vertex to the source vertex. This is maintained by simply adding the statement

+
predecessors[v] = u
+
+

inside of the if statement's body above.

+

Example

+

For the following weighted directed graph:

+

+

let's compute the shortest paths from vertex s. First, we prepare our weights and predecessors structures thusly:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
weightspredecessors
weights[s] = 0predecessors[s] = 1
weights[t] = ∞predecessors[t] = ø
weights[x] = ∞predecessors[x] = ø
weights[y] = ∞predecessors[y] = ø
weights[z] = ∞predecessors[z] = ø
+

Here are their states after each relaxation iteration (each iteration is a pass over all edges, and there are 4 iterations total for this graph):

+
Iteration 1:
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
weightspredecessors
weights[s] = 0predecessors[s] = s
weights[t] = 6predecessors[t] = s
weights[x] = 4predecessors[x] = y
weights[y] = 7predecessors[y] = s
weights[z] = 2predecessors[z] = t
+
Iteration 2:
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
weightspredecessors
weights[s] = 0predecessors[s] = s
weights[t] = 2predecessors[t] = x
weights[x] = 4predecessors[x] = y
weights[y] = 7predecessors[y] = s
weights[z] = 2predecessors[z] = t
+
Iteration 3:
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
weightspredecessors
weights[s] = 0predecessors[s] = s
weights[t] = 2predecessors[t] = x
weights[x] = 4predecessors[x] = y
weights[y] = 7predecessors[y] = s
weights[z] = -2predecessors[z] = t
+
Iteration 4:
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
weightspredecessors
weights[s] = 0predecessors[s] = s
weights[t] = 2predecessors[t] = x
weights[x] = 4predecessors[x] = y
weights[y] = 7predecessors[y] = s
weights[z] = -2predecessors[z] = t
+

Negative weight cycles

+

An additional useful property of the solution structure is that it can answer whether or not a negative weight cycle exists in the graph and is reachable from the source vertex. A negative weight cycle is a cycle whose sum of edge weights is negative. This means that shortest paths are not well defined in the graph from the specified source, because you can decrease the weight of a path by reentering the cycle, pushing the path's weight towards -∞. After fully relaxing the paths, simply running a check over each edge e = (u, v) to see if the weight of the shortest path to v is greater than the path to u, plus the edge weight itself, signals that the edge has a negative weight and would decrease the shortest path's weight further. Since we know we've already performed the relaxations enough times according to the intuition stated above, we can safely assume this further decrease of weight will continue infinitely.

+
Example
+

For this example, we try to compute the shortest paths from a:

+

+

The cycle a->t->s->a has a total edge weight of -9, therefore shortest paths for a>t and a>s are not well-defined. a~>b is also not well-defined because b->t->s is also a negative weight cycle.

+

This is confirmed after running the relaxation loop, and checking all the edges as mentioned above. For this graph, we would have after relaxation:

+ + + + + + + + + + + + + + + + + + + + +
weights
weights[a] = -5
weights[b] = -5
weights[s] = -18
weights[t] = -3
+

One of the edge checks we would perform afterwards would be the following:

+
e = (s, a)
+e.weight = 4
+weight[a] > weight[s] + e.weight => -5 > -18 + 4 => -5 > -14 => true
+
+

Because this check is true, we know the graph has a negative weight cycle reachable from a.

+

Complexity

+

The relaxation step requires constant time (O(1)) as it simply performs comparisons. That step is performed once per edge (Θ(|E|)), and the edges are iterated over |V|-1 times. This would mean a total complexity of Θ(|V||E|), but there is an optimization we can make: if the outer loop executes and no changes are made to the recorded weights, we can safely terminate the relaxation phase, which means it may execute in O(|V|) steps instead of Θ(|V|) steps (that is, the best case for any size graph is actually a constant number of iterations; the worst case is still iterating |V|-1 times).

+

The check for negative weight cycles at the end is O(|E|) if we return once we find a hit. To find all negative weight cycles reachable from the source vertex, we'd have to iterate Θ(|E|) times (we currently do not attempt to report the cycles, we simply return a nil result if such a cycle is present).

+

The total running time of Bellman-Ford is therefore O(|V||E|).

+

TODO

+ +

References

+ +

Written for Swift Algorithm Club by Andrew McKnight

+ + diff --git a/Singly Linked List/index.html b/Singly Linked List/index.html new file mode 100644 index 000000000..099a65d92 --- /dev/null +++ b/Singly Linked List/index.html @@ -0,0 +1,122 @@ + + + Singly Linked List + + + +

Singly-Linked List

+

How is this different to the Linked List implementation?

+

The existing Linked list implementation represents the same concept. However, the existing implementation has reference semantics and does not conform to the Collection protocol implemented in the Swift's standard Library. What SinglyLinkedList aims to contribute is a more idiomatic Swift implementation, that uses value semantics and copy-on-write as well as conforms to the collection protocol.

+

Conceptual representation

+

A Singly linked list is a non-contiguous sequence of data items in memory. Each element links to the next via a memory reference. Additionally, this implementation keeps track of the last element, which can be retrived in order O(1). However, the list can only be traversed from head to tail.

+
+--------+    +--------+    +--------+    +--------+
+|        |    |        |    |        |    |        |
+| node 0 |--->| node 1 |--->| node 2 |--->| node 3 |---> nil
+|        |    |        |    |        |    |        |
++--------+    +--------+    +--------+    +--------+
+^                                         ^
+|                                         |
+Head                                      Tail
+
+

Each element in the list is represented with an instance of SinglyLinkedListNode class, which basically contains some data and a reference of optional type to the next node, which means that the last node's next reference is nil.

+

In-memory representation

+

In Swift, data types can have value or reference semantics. This implementation of a singly-linked list uses value semantics. Support for copy-on-write has been added in order to improve performance and delay copying the elements of the array until strictly necessary.

+

The image below shows how initially, after variable l2 is assigned l1, a new instance of the struct SinglyLinkedList is created. Nevertheless, the indirect storage is still shared as indicated by the references that both l1 and l2 have pointing to a common area in memory.

+

alt text

+

Once a mutating operation happens --for example on l2 to append a new element--, then the indirect storage and all nodes in the list is actually copied and the references from l1 and l2 are updated. This is ilustrated in the following figure.

+

alt text

+

Implementation details

+
    +
  1. Direct access to the tail in O(1) by keeping a reference that gets updated only when an operation to the list modifies the tail.
    2. +
  2. Value semantics. This implementation of a singly-linked list uses a struct. When the list struct is assigned into another variable or passed as an argument to a function, a copy of the struct is made.
    3. +
  3. Copy-on write. Instances of SinglyLinkedList have an internal reference to an indirect storage allocated in the heap. When a copy of the list is made (according to its value semantics) the indirect storage is initialy shared between the copies. This means that a potentially large list can be accessed to read values in it without an expensive copy having to take place. However, as soon as there is a write access to the indirect storage when there are more than one instances referencing to it, a copy will be performed to guarantee value semantics.
    4. +
  4. Conformance to the Collection protocol.
    5. +
+

Performance of linked lists

+

EDITION

+ +

SEARCH

+ +

Conforming to the Collection protocol

+

Collections are sequences, therefore the first step is to conform to the Sequence protocol.

+
extension SinglyLinkedList: Sequence {
+    public func makeIterator() -> SinglyLinkedListForwardIterator<T> {
+        return SinglyLinkedListForwardIterator(head: self.storage.head)
+    }
+}
+
+

We have used SinglyLinkedListForwardIterator an iterator class to keep track of the progress while iterating the structure:

+
public struct SinglyLinkedListForwardIterator<T> : IteratorProtocol {
+
+    public typealias Element = T
+
+    private(set) var head: SinglyLinkedListNode<T>?
+
+    mutating public func next() -> T? {
+        let result = self.head?.value
+        self.head = self.head?.next
+        return result
+    }
+}
+
+

Next, a collection needs to be indexable. Indexes are implemented by the data structure class. We have encapsulated this knowledge in instances of the class SinglyLinkedListIndex.

+
public struct SinglyLinkedListIndex<T>: Comparable {
+    fileprivate let node: SinglyLinkedListNode<T>?
+    fileprivate let tag: Int
+
+    public static func==<T>(lhs: SinglyLinkedListIndex<T>, rhs: SinglyLinkedListIndex<T>) -> Bool {
+        return (lhs.tag == rhs.tag)
+    }
+
+    public static func< <T>(lhs: SinglyLinkedListIndex<T>, rhs: SinglyLinkedListIndex<T>) -> Bool {
+        return (lhs.tag < rhs.tag)
+    }
+}
+
+

Finally, the methods in the collection to manipulate indexes are implemented below:

+ +
extension SinglyLinkedList : Collection {
+
+    public typealias Index = SinglyLinkedListIndex<T>
+
+    public var startIndex: Index {
+        get {
+            return SinglyLinkedListIndex<T>(node: self.storage.head, tag: 0)
+        }
+    }
+
+public var endIndex: Index {
+    get {
+        if let h = self.storage.head {
+        let (_, numberOfElements) = findTail(in: h)
+            return SinglyLinkedListIndex<T>(node: h, tag: numberOfElements)
+        } else {
+            return SinglyLinkedListIndex<T>(node: nil, tag: self.startIndex.tag)
+        }
+    }
+}
+
+public subscript(position: Index) -> T {
+    get {
+        return position.node!.value
+    }
+}
+
+public func index(after idx: Index) -> Index {
+    return SinglyLinkedListIndex<T>(node: idx.node?.next, tag: idx.tag+1)
+    }
+}
+
+

Conforming to the Collection protocol allows our class SinglyLinkedList to take adventage of all the collection methods included in the Stardard Library.

+

Written by Borja Arias Drake

+ + diff --git a/Skip-List/index.html b/Skip-List/index.html new file mode 100644 index 000000000..a5d827a2e --- /dev/null +++ b/Skip-List/index.html @@ -0,0 +1,93 @@ + + + Skip-List + + + +

Skip List

+

Skip List is a probablistic data-structure with same logarithmic time bound and
+efficiency as AVL/ or Red-Black tree and provides a clever compromise to
+efficiently support search and update operations and is relatively simpler to
+implement compared to other map data structures.

+

A skip list S consists of series of sorted linked lists {L0, ..., Ln},
+layered hierarchicaly and each layer L stores a subset of items in layer L0
+in incremental order. The items in layers {L1, ... Ln} are chosen at random
+based on a coin flipping function with probability 1/2 . For traversing, every
+item in a layer hold references to the node below and the next node. This
+layers serve as express lanes to the layer underneath them, effectively making
+fast O(log n) searching possible by skipping lanes and reducing travel distance
+and in worse case searching degrades to O (n), as expected with regular linked
+list.

+

For a skip list S:

+
    +
  1. List L0 contains every inserted item.
    2. +
  2. For lists {L1, ..., Ln}, Li contains a randomly generated subset of the
    +items in Li-1
    3. +
  3. Height is determined by coin-flipping.
    4. +
+

Schematic view
+Figure 1

+

#Searching

+

Searching for element N starts by traversing from top most layer Ln until
+L0.

+

Our objective is to find an element K such that its value at the rightmost
+position of current layer, is less-than target item and its subsequent node has
+a greater-equal value or nil ( K.key < N.key <= (K.next.key or nil) ). if
+value of K.next is equal to N, search is terminated and we return K.next,
+otherwise drop underneath using K.down to the node below ( at layer Ln-1 ) and
+repeat the process until L0 where K.down is nil which indicates that level
+is L0 and item doesn't exists.

+

###Example:

+

Inserting first element

+

#Inserting

+

Inserting element N has a similar process as searching. It starts by
+traversing from top most layer Ln until L0. We need to keep track of our
+traversal path using a stack. It helps us to traverse the path upward when
+coin-flipping starts, so we can insert our new element and update references to
+it.

+

Our objective is to find a element K such that its value at the rightmost
+position of layer Ln, is less-than new item and its subsequent node has a
+greater-equal value or nil ( K.key < N.key < (K.next.key or nil) ). Push
+element K to the stack and with element K, go down using K.down to the
+node below ( at layer Ln-1 ) and repeat the process ( forward searching ) up
+until L0 where K.down is nil which indicates that level is L0. We
+terminate the process when K.down is nil.

+

At L0, N can be inserted after K.

+

Here is the interesting part. We use coin flipping function to randomly create
+layers.

+

When coin flip function returns 0, the whole process is finished but when
+returns 1, there are two possibilities:

+
    +
  1. Stack is empty ( Level is L0 /- Ln or at uninitialized stage)
    2. +
  2. Stack has items ( traversing upward is possible )
    3. +
+

In case 1:

+

A new layer M* is created with a head node NM referencing head node of layer
+below and NM.next referencing new element N. New element N referecing
+element N at previous layer.

+

In case 2:

+

repeat until stack is empty Pop an item F from stack and update the references
+accordingly. F.next will be K.next and K.next will be F

+

when stack is empty Create a new layer consisintg of a head node NM
+referencing head node of layer below and NM.next referencing new element
+N. New element N referencing element N at previous layer.

+

###Example:

+

Inserting 13. with coin flips (0)

+

Inserting first element
+Inserting first element
+Inserting first element
+Inserting first element
+Inserting first element

+

Inserting 20. with 4 times coin flips (1)
+Inserting first element
+Inserting first element
+Inserting first element
+Inserting first element

+

#Removing

+

Removing works similar to insert procedure.

+

TODO

+

#See also

+

Skip List on Wikipedia

+

Written for Swift Algorithm Club by Mike Taghavi

+ + diff --git a/Slow Sort/index.html b/Slow Sort/index.html new file mode 100644 index 000000000..def1e96ce --- /dev/null +++ b/Slow Sort/index.html @@ -0,0 +1,62 @@ + + + Slow Sort + + + +

Slow Sort

+

Goal: Sort an array of numbers from low to high (or high to low).

+

You are given an array of numbers and need to put them in the right order. The insertion sort algorithm works as follows:

+

We can decompose the problem of sorting n numbers in ascending order into

+
    +
  1. find the maximum of the numbers
    2. +
  2. find the maximum of the first n/2 elements
    3. +
  3. find the maximum of the remaining n/2 elements
    4. +
  4. find the largest of those two maxima
    5. +
  5. sorting the remaining ones
    6. +
+

The code

+

Here is an implementation of slow sort in Swift:

+
public func slowsort(_ i: Int, _ j: Int) {
+    if i>=j {
+        return
+    }
+    let m = (i+j)/2
+    slowsort(i,m)
+    slowsort(m+1,j)
+    if numberList[j] < numberList[m] {
+        let temp = numberList[j]
+        numberList[j] = numberList[m]
+        numberList[m] = temp
+    }
+    slowsort(i,j-1)
+}
+

Performance

+ + + + + + + + + + + + + + + + + + + + + +
CasePerformance
Worstslow
BestO(n^(log(n)/(2+e))))
AverageO(n^(log(n)/2))
+

See also

+

Slow Sort explanation in the Internet

+

Written for Swift Algorithm Club by Lukas Schramm

+

(used the Insertion Sort Readme as template)

+ + diff --git a/Sorted Set/index.html b/Sorted Set/index.html new file mode 100644 index 000000000..97f8c5a3e --- /dev/null +++ b/Sorted Set/index.html @@ -0,0 +1,243 @@ + + + Sorted Set + + + +

Sorted Set

+

Sorted Array Version

+

An Sorted Set is a collection of unique items in sorted order. Items are usually sorted from least to greatest.

+

The Sorted Set data type is a hybrid of:

+ +

It's important to keep in mind that two items can have the same value but still may not be equal. For example, we could define "a" and "z" to have the same value (their lengths), but clearly "a" != "z".

+

Why use an sorted set?

+

Sorted Sets should be considered when you need to keep your collection sorted at all times, and you do lookups on the collection much more frequently than inserting or deleting items. Many of the lookup operations for an Sorted Set are O(1).

+

A good example would be keeping track of the rankings of players in a scoreboard (see example 2 below).

+

These are sorted sets

+

A set of integers:

+
[1, 2, 3, 6, 8, 10, 1000]
+
+

A set of strings:

+
["a", "is", "set", "this"]
+
+

The "value" of these strings could be their text content, but also for example their length.

+

These are not sorted sets

+

This set violates the property of uniqueness:

+
[1, 1, 2, 3, 5, 8]
+
+

This set violates the sorted property:

+
[1, 11, 2, 3]
+
+

The code

+

We'll start by creating our internal representation for the Sorted Set. Since the idea of a set is similar to that of an array, we will use an array to represent our set. Furthermore, since we'll need to keep the set sorted, we need to be able to compare the individual elements. Thus, any type must conform to the Comparable Protocol.

+
public struct SortedSet<T: Comparable> {
+  private var internalSet = [T]()
+
+  // Returns the number of elements in the SortedSet.
+  public var count: Int {
+    return internalSet.count
+  }
+  ...
+

Lets take a look at the insert() function first. This first checks if the item already exists in the collection. If so, it returns and does not insert the item. Otherwise, it will insert the item through straightforward iteration.

+
  public mutating func insert(_ item: T){
+    if exists(item) {
+      return  // don't add an item if it already exists
+    }
+
+    // Insert new the item just before the one that is larger.
+    for i in 0..<count {
+      if internalSet[i] > item {
+        internalSet.insert(item, at: i)
+        return
+      }
+    }
+
+    // Append to the back if the new item is greater than any other in the set.
+    internalSet.append(item)
+  }
+

As we'll see later on, checking if the item is already in the set has an efficiency of O(log(n) + k) where k is the number of items with the same value as the item we are inserting.

+

To insert the new item, the for loop starts from the beginning of the array, and checks to see if each item is larger than the item we want to insert. Once we find such an item, we insert the new one into its place. This shifts the rest of the array over to the right by 1 position. This loop is at worst O(n).

+

The total performance of the insert() function is therefore O(n).

+

Next up is the remove() function:

+
  public mutating func remove(_ item: T) {
+    if let index = index(of: item) {
+      internalSet.remove(at: index)
+    }
+  }
+

First this checks if the item exists and then removes it from the array. Because of the removeAtIndex() function, the efficiency for remove is O(n).

+

The next function is indexOf(), which takes in an object of type T and returns the index of the corresponding item if it is in the set, or nil if it is not. Since our set is sorted, we can use a binary search to quickly search for the item.

+
  public func index(of item: T) -> Int? {
+    var leftBound = 0
+    var rightBound = count - 1
+
+    while leftBound <= rightBound {
+      let mid = leftBound + ((rightBound - leftBound) / 2)
+
+      if internalSet[mid] > item {
+        rightBound = mid - 1
+      } else if internalSet[mid] < item {
+        leftBound = mid + 1
+      } else if internalSet[mid] == item {
+        return mid
+      } else {
+      	// see below
+      }
+    }
+    return nil
+  }
+
+

Note: If you are not familiar with the concept of binary search, we have an article that explains all about it.

+
+

However, there is an important issue to deal with here. Recall that two objects can be unequal yet still have the same "value" for the purposes of comparing them. Since a set can contain multiple items with the same value, it is important to check that the binary search has landed on the correct item.

+

For example, consider this sorted set of Player objects. Each Player has a name and a number of points:

+
[ ("Bill", 50), ("Ada", 50), ("Jony", 50), ("Steve", 200), ("Jean-Louis", 500), ("Woz", 1000) ]
+
+

We want the set to be sorted by points, from low to high. Multiple players can have the same number of points. The name of the player is not important for this ordering. However, the name is important for retrieving the correct item.

+

Let's say we do indexOf(bill) where bill is player object ("Bill", 50). If we did a traditional binary search we'd land on index 2, which is the object ("Jony", 50). The value 50 matches, but it's not the object we're looking for!

+

Therefore, we also need to check the items with the same value to the right and left of the midpoint. The code to check the left and right side looks like this:

+
        // Check to the right.
+        for j in mid.stride(to: count - 1, by: 1) {
+          if internalSet[j + 1] == item {
+            return j + 1
+          } else if internalSet[j] < internalSet[j + 1] {
+            break
+          }
+        }
+
+        // Check to the left.
+        for j in mid.stride(to: 0, by: -1) {
+          if internalSet[j - 1] == item {
+            return j - 1
+          } else if internalSet[j] > internalSet[j - 1] {
+            break
+          }
+        }
+
+        return nil
+

These loops start at the current mid value and then look at the neighboring values until we've found the correct object.

+

The combined runtime for indexOf() is O(log(n) + k) where n is the length of the set, and k is the number of items with the same value as the one that is being searched for.

+

Since the set is sorted, the following operations are all O(1):

+
  // Returns the 'maximum' or 'largest' value in the set.
+  public func max() -> T? {
+    return count == 0 ? nil : internalSet[count - 1]
+  }
+
+  // Returns the 'minimum' or 'smallest' value in the set.
+  public func min() -> T? {
+    return count == 0 ? nil : internalSet[0]
+  }
+
+  // Returns the k-th largest element in the set, if k is in the range
+  // [1, count]. Returns nil otherwise.
+  public func kLargest(_ k: Int) -> T? {
+    return k > count || k <= 0 ? nil : internalSet[count - k]
+  }
+
+  // Returns the k-th smallest element in the set, if k is in the range
+  // [1, count]. Returns nil otherwise.
+  public func kSmallest(_ k: Int) -> T? {
+    return k > count || k <= 0 ? nil : internalSet[k - 1]
+  }
+

Examples

+

Below are a few examples that can be found in the playground file.

+

Example 1

+

Here we create a set with random Integers. Printing the largest/smallest 5 numbers in the set is fairly easy.

+
// Example 1 with type Int
+var mySet = SortedSet<Int>()
+
+// Insert random numbers into the set
+for _ in 0..<50 {
+  mySet.insert(randomNum(50, max: 500))
+}
+
+print(mySet)
+
+print(mySet.max())
+print(mySet.min())
+
+// Print the 5 largest values
+for k in 1...5 {
+  print(mySet.kLargest(k))
+}
+
+// Print the 5 lowest values
+for k in 1...5 {
+  print(mySet.kSmallest(k))
+}
+

Example 2

+

In this example we take a look at something a bit more interesting. We define a Player struct as follows:

+
public struct Player: Comparable {
+  public var name: String
+  public var points: Int
+}
+

The Player also gets its own == and < operators. The < operator is used to determine the sort order of the set, while == determines whether two objects are really equal.

+

Note that == compares both the name and the points:

+
func ==(x: Player, y: Player) -> Bool {
+  return x.name == y.name && x.points == y.points
+}
+
+

But < only compares the points:

+
func <(x: Player, y: Player) -> Bool {
+  return x.points < y.points
+}
+

Therefore, two Players can each have the same value (the number of points), but are not guaranteed to be equal (they can have different names).

+

We create a new set and insert 20 random players. The Player() constructor gives each player a random name and score:

+
var playerSet = SortedSet<Player>()
+
+// Populate the set with random players.
+for _ in 0..<20 {
+  playerSet.insert(Player())
+}
+

Insert another player:

+
var anotherPlayer = Player()
+playerSet.insert(anotherPlayer)
+

Now we use the indexOf() function to find out what rank anotherPlayer is.

+
let level = playerSet.count - playerSet.indexOf(anotherPlayer)!
+print("\(anotherPlayer.name) is ranked at level \(level) with \(anotherPlayer.points) points")
+

Example 3

+

The final example demonstrates the need to look for the right item even after the binary search has completed.

+

We insert 9 players into the set:

+
var repeatedSet = SortedSet<Player>()
+
+repeatedSet.insert(Player(name: "Player 1", points: 100))
+repeatedSet.insert(Player(name: "Player 2", points: 100))
+repeatedSet.insert(Player(name: "Player 3", points: 100))
+repeatedSet.insert(Player(name: "Player 4", points: 100))
+repeatedSet.insert(Player(name: "Player 5", points: 100))
+repeatedSet.insert(Player(name: "Player 6", points: 50))
+repeatedSet.insert(Player(name: "Player 7", points: 200))
+repeatedSet.insert(Player(name: "Player 8", points: 250))
+repeatedSet.insert(Player(name: "Player 9", points: 25))
+

Notice how several of these players have the same value of 100 points.

+

The set looks something like this:

+
[Player 9, Player 6, Player 1, Player 2, Player 3, Player 4, Player 5, Player 7, Player 8]
+
+

The next line looks for Player 2:

+
print(repeatedSet.index(of: Player(name: "Player 2", points: 100)))
+

After the binary search finishes, the value of mid is at index 5:

+
[Player 9, Player 6, Player 1, Player 2, Player 3, Player 4, Player 5, Player 7, Player 8]
+                                                      mid
+
+

However, this is not Player 2. Both Player 4 and Player 2 have the same points, but a different name. The binary search only looked at the points, not the name.

+

But we do know that Player 2 must be either to the immediate left or the right of Player 4, so we check both sides of mid. We only need to look at the objects with the same value as Player 4. The others are replaced by X:

+
[X, X, Player 1, Player 2, Player 3, Player 4, Player 5, X, X]
+                                       mid
+
+

The code then first checks on the right of mid (where the * is):

+
[X, X, Player 1, Player 2, Player 3, Player 4, Player 5, X, X]
+                                       mid        *
+
+

The right side did not contain the item, so we look at the left side:

+
[X, X, Player 1, Player 2, Player 3, Player 4, Player 5, X, X]
+                              *        mid        
+
+[X, X, Player 1, Player 2, Player 3, Player 4, Player 5, X, X]
+                    *                  mid        
+
+

Finally, we've found Player 2, and return index 3.

+

Written By Zain Humayun

+ + diff --git a/Sparse Table/index.html b/Sparse Table/index.html new file mode 100644 index 000000000..d6de81cb3 --- /dev/null +++ b/Sparse Table/index.html @@ -0,0 +1,272 @@ + + + Sparse Table + + + +

Sparse Table

+

I'm excited to present Sparse Tables. Despite being somewhat niche, Sparse Tables are simple to implement and extremely powerful.

+

The Problem

+

Let's suppose:

+ +

[*] where f is also "idempotent". Don't worry, I'll explain this in a moment.

+

Our task is as follows:

+ +

For example, if we have an array of numbers:

+
var a = [ 20, 3, -1, 101, 14, 29, 5, 61, 99 ]
+

and our function f is the min function.

+

Then we may be given a query for interval [3, 8). That means we look at the elements:

+
101, 14, 29, 5, 61
+
+

because these are the elements of a with indices
+that lie in our range [3, 8) – elements from index 3 up to, but not including, index 8.
+We then we pass all of these numbers into the min function,
+which takes the minimum. The answer to the query is 5, because that's the result of min(101, 14, 29, 5, 61).

+

Imagine we have millions of these queries to process.

+
+ +
+

And our array is very large. Here, let's say Q = 1000000 and N = 500000. Both numbers are huge. We want to make sure that we can answer each query really quickly, or else the number of queries will overwhelm us!

+

So that's the problem.

+

The naive solution to this problem is to perform a for loop
+to compute the answer for each query. However, for very large Q and very large N this
+will be too slow. We can speed up the time to compute the answer by using a data structure called
+a Sparse Table. You'll notice that so far, our problem is exactly the same as that of the Segment Tree
+(assuming you're familiar). However! ... there's one crucial difference between Segment Trees
+and Sparse Tables ... and it concerns our choice of f.

+

A small gotcha ... Idempotency

+

Suppose we wanted to find the answer to [A, D).
+And we already know the answer to two ranges [A, B) and [C, D).
+And importantly here, ... these ranges overlap!! We have C < B.

+

Overlapping ranges

+

So what? Well, for f = minimum function, we can take our answers for [A, B) and [C, D)
+and combine them!
+We can just take the minimum of the two answers: result = min(x1, x2) ... voilà!, we have the minimum for [A, D).
+It didn't matter that the intervals overlap - we still found the correct minimum.
+But now suppose f is the addition operation +. Ok, so now we're taking sums over ranges.
+If we tried the same approach again, it wouldn't work. That is,
+if we took our answers for [A, B) and [C, D)
+and added them together we'd get a wrong answer for [A, D).
+Why? Well, we'd have counted some elements twice because of the overlap.

+

Later, we'll see that in order to answer queries, Sparse Tables use this very technique.
+They combine answers in the same way as shown above. Unfortunately this means
+we have to exclude certain binary operators from being f, including +, *, XOR, ...
+because they don't work with this technique.
+In order to get the best speed of a Sparse Table,
+we need to make sure that the f we're using is an idempotent binary operator.
+Mathematically, these are operators that satisfy f(x, x) = x for all possible x that could be in a.
+Practically speaking, these are the only operators that work; allowing us to combine answers from overlapping ranges.
+Examples of idempotent f's are min, max, gcd, boolean AND, boolean OR, bitwise AND and bitwise OR.
+Note that for Segment Trees, f does not have to be idempotent. That's the crucial difference between
+Segment Trees and Sparse Tables.

+

Phew! Now that we've got that out of the way, let's dive in!

+

Structure of a Sparse Table

+

Let's use f = min and use the array:

+
var a = [ 10, 6, 5, -7, 9, -8, 2, 4, 20 ]
+

In this case, the Sparse Table looks like this:

+

Sparse Table

+

What's going on here? There seems to be loads of intervals.
+Correct! Sparse tables are preloaded with the answers for lots of queries [l, r).
+Here's the idea. Before we process our Q queries, we'll pre-populate our Sparse Table table
+with answers to loads of queries;
+making it act a bit like a cache. When we come to answer one of our queries, we can break the query
+down into smaller "sub-queries", each having an answer that's already in the cache.
+We lookup the cached answers for the sub-queries in
+table in constant time
+and combine the answers together
+to give the overall answer to the original query in speedy time.

+

The problem is, we can't store the answers for every single possible query that we could ever have ...
+or else our table would be too big! After all, our Sparse Table needs to be sparse. So what do we do?
+We only pick the "best" intervals to store answers for. And as it turns out, the "best" intervals are those
+that have a width that is a power of two!

+

For example, the answer for the query [10, 18) is in our table
+because the interval width: 18 - 10 = 8 = 2**3 is a power of two (** is the exponentiation operator).
+Also, the answer for [15, 31) is in our table because its width: 31 - 15 = 16 = 2**4 is again a power of two.
+However, the answer for [1, 6) is not in there because the interval's width: 6 - 1 = 5 is not a power of two.
+Consequently, we don't store answers for all possible intervals that fit inside a –
+only the ones with a width that is a power of two.
+This is true irrespective of where the interval starts within a.
+We'll gradually see that this approach works and that relatively speaking, it uses very little space.

+

A Sparse Table is a table where table[w][l] contains the answer for [l, l + 2**w).
+It has entries table[w][l] where:

+ +

Some examples:

+ +

Sparse Table

+

A Sparse Table can be implemented using a two-dimentional array.

+
public class SparseTable<T> {
+  private var table: [[T]]
+
+  public init(array: [T], function: @escaping (T, T) -> T, defaultT: T) {
+      table = [[T]](repeating: [T](repeating: defaultT, count: N), count: W)
+  }
+  // ...    
+}
+

Building a Sparse Table

+

To build a Sparse Table, we compute each table entry starting from the bottom-left and moving up towards
+the top-right (in accordance with the diagram).
+First we'll compute all the intervals for w = 0, then compute all the intervals
+and for w = 1 and so on. We'll continue up until w is big enough such that our intervals are can cover at least half the array.
+For each w, we compute the interval for l = 0, 1, 2, 3, ... until we reach N.
+This is all achieved using a double for-in loop:

+
for w in 0..<W {
+  for l in 0..<N {
+    // compute table[w][l]
+  }
+}
+

To compute table[w][l]:

+ +

Sparse Table

+

For example for a = [ 10, 6, 5, -7, 9, -8, 2, 4, 20 ] and f = min:

+ +

Sparse Table

+
public init(array: [T], function: @escaping (T, T) -> T, defaultT: T) {
+ let N = array.count
+ let W = Int(ceil(log2(Double(N))))
+ table = [[T]](repeating: [T](repeating: defaultT, count: N), count: W)
+ self.function = function
+ self.defaultT = defaultT
+
+ for w in 0..<W {
+   for l in 0..<N {
+     if w == 0 {
+       table[w][l] = array[l]
+     } else {
+       let first = self.table[w - 1][l]
+       let secondIndex = l + (1 << (w - 1))
+       let second = ((0..<N).contains(secondIndex)) ? table[w - 1][secondIndex] : defaultT
+       table[w][l] = function(first, second)
+     }
+   }
+ }
+}
+

Building a Sparse Table takes O(NlogN) time.
+The table itself uses O(NlgN) additional space.

+

Getting Answers to Queries

+

Suppose we've built our Sparse Table. And now we're going to process our Q queries.
+Here's where our work pays off.

+

Let's suppose f = min and we have:

+
var a = [ 10, 6, 5, -7, 9, -8, 2, 4, 20 ]
+

And we have a query [3, 9).

+
    +
  1. +

    First let's find the largest power of two that fits inside [3, 9). Our interval has width 9 - 3 = 6. So the largest power of two that fits inside is four.

    +
    2. +
  2. +

    We create two new queries of [3, 7) and [5, 9) that have a width of four.
    +And, we arrange them so that to that they span the whole interval without leaving any gaps.
    +Sparse Table

    +
    3. +
  3. +

    Because these two intervals have a width that is exactly a power of two we can lookup their answers in the Sparse Table using the
    +entries for w = 2. The answer to [3, 7) is given by table[2][3], and the answer to [5, 9) is given by table[2][5].
    +We compute and return min(table[2][3], table[2][5]). This is our final answer! 🎉. Although the two intervals overlap, it doesn't matter because the f = min we originally chose is idempotent.

    +
    4. +
+

In general, for each query: [l, r) ...

+
    +
  1. +

    Find W, by looking for the largest width that fits inside the interval that's also a power of two. Let largest such width = 2**W.

    +
    2. +
  2. +

    Form two sub-queries of width 2**W and arrange them to that they span the whole interval without leaving gaps.
    +To guarantee there are no gaps, we need to align one half to the left and the align other half to the right.
    +Sparse Table

    +
    3. +
  3. +

    Compute and return f(table[W][l], table[W][r - 2**W]).

    +
    public func query(from l: Int, until r: Int) -> T {
+  let width = r - l
+  let W = Int(floor(log2(Double(width))))
+  let lo = table[W][l]
+  let hi = table[W][r - (1 << W)]
+  return function(lo, hi)
+}
    +
    4. +
+

Finding answers to queries takes O(1) time.

+

Analysing Sparse Tables

+ +

Comparison with Segment Trees

+ +

[†] Although technically, it's possible to rewrite the query method
+to add support for non-idempotent functions. But in doing so, we'd bump up the time up from O(1) to O(lgn),
+completely defeating the original purpose of Sparse Tables - supporting lightening quick queries.
+In such a case, we'd be better off using a Segment Tree (or a Fenwick Tree)

+

Summary

+

That's it! See the playground for more examples involving Sparse Tables.
+You'll see examples for: min, max, gcd, boolean operators and logical operators.

+

See also

+ +

Written for Swift Algorithm Club by James Lawson

+ + diff --git a/Splay Tree/index.html b/Splay Tree/index.html new file mode 100644 index 000000000..1e05cfcea --- /dev/null +++ b/Splay Tree/index.html @@ -0,0 +1,218 @@ + + + Splay Tree + + + +

Splay Tree

+

Splay tree is a data structure, structurally identitical to a balanced binary search tree. Every operation performed on a Splay Tree causes a readjustment in order to provide fast access to recently operated values. On every access, the tree is rearranged and the node accessed is moved to the root of the tree using a set of specific rotations, which together are referred to as Splaying.

+

Rotations

+

There are 3 types of rotations that can form an Splaying:

+ +

Zig-Zig

+

Given a node a if a is not the root, and a has a child b, and both a and b are left children or right children, a Zig-Zig is performed.

+

Case both nodes right children

+

ZigZigCase1

+

Case both nodes left children

+

ZigZigCase2

+

IMPORTANT is to note that a ZigZig performs first the rotation of the middle node with its parent (call it the grandparent) and later the rotation of the remaining node (grandchild). Doing that helps to keep the trees balanced even if it was first created by inserted a sequence of increasing values (see below worst case scenario followed by an explanation about why ZigZig rotates first to the grandparent).

+

Zig-Zag

+

Given a node a if a is not the root, and a has a child b, and b is the left child of a being the right child (or the opposite), a Zig-Zag is performed.

+

Case right - left

+

ZigZagCase1

+

Case left - right

+

ZigZagCase2

+

IMPORTANT A ZigZag performs first the rotation of the grandchild node and later the same node with its new parent again.

+

Zig

+

A Zig is performed when the node a to be rotated has the root as parent.

+

ZigCase

+

Splaying

+

Splaying consists in making so many rotations as needed until the node affected by the operation is at the top and becomes the root of the tree.

+
while (node.parent != nil) {
+    operation(forNode: node).apply(onNode: node)
+}
+
+

Where operation returns the required rotation to be applied.

+
public static func operation<T>(forNode node: Node<T>) -> SplayOperation {
+    
+    if let parent = node.parent, let _ = parent.parent {
+        if (node.isLeftChild && parent.isRightChild) || (node.isRightChild && parent.isLeftChild) {
+            return .zigZag
+        }
+        return .zigZig
+    }
+    return .zig
+}
+
+

During the applying phase, the algorithms determines which nodes are involved depending on the rotation to be applied and proceeding to re-arrange the node with its parent.

+
public func apply<T>(onNode node: Node<T>) {
+    switch self {
+    case .zigZag:
+        assert(node.parent != nil && node.parent!.parent != nil, "Should be at least 2 nodes up in the tree")
+        rotate(child: node, parent: node.parent!)
+        rotate(child: node, parent: node.parent!)
+
+    case .zigZig:
+        assert(node.parent != nil && node.parent!.parent != nil, "Should be at least 2 nodes up in the tree")
+        rotate(child: node.parent!, parent: node.parent!.parent!)
+        rotate(child: node, parent: node.parent!)
+    
+    case .zig:
+        assert(node.parent != nil && node.parent!.parent == nil, "There should be a parent which is the root")
+        rotate(child: node, parent: node.parent!)
+    }
+}
+
+

Operations on an Splay Tree

+

Insertion

+

To insert a value:

+ +

Deletion

+

To delete a value:

+ +

Search

+

To search a value:

+ +

Minimum and maximum

+ +

Examples

+

Example 1

+

Lets suppose a find(20) operation was performed and now the values 20 needs to be splayed to the root.
+The sequence of steps will be the following:

+
    +
  1. Since we are in a ZigZig case, we need to rotate 9 to 4
    2. +
+

ZiggEx1

+
    +
  1. We got the following tree after the first rotation:
    2. +
+

ZiggEx2

+
    +
  1. And finally the 20 is rotated to the 9
    2. +
+

ZiggEx3

+

Example 2

+

Now suppose a insert(7) operation was performed and we're in a ZigZag case.

+
    +
  1. First 7 is rotated to 9
    2. +
+

ZigggEx21

+
    +
  1. And the result tree is:
    2. +
+

ZigggEx22

+
    +
  1. Finally 7 is rotated to 4
    2. +
+

ZigggEx23

+

Advantages

+

Splay trees provide an efficient way to quickly access elements that are frequently requested. This characteristic makes then a good choice to implement, for example, caches or garbage collection algorithms, or in any other problem involving frequent access to a certain numbers of elements from a data set.

+

Disadvantages

+

Splay tree are not perfectly balanced always, so in case of accessing all the elements in the tree in an increasing order, the height of the tree becomes n.

+

Time complexity

+ + + + + + + + + + + + + + + + + +
CasePerformance
AverageO(log n)
Worstn
+

With n being the number of items in the tree.

+

An example of the Worst Case Performance

+

Suppose the a sequence of consecutive values are inserted in an Splay Tree.
+Let's take for example [1,2,3,4,5,6,7,8].

+

The tree construction will be like following:

+
    +
  1. +

    Insert the number 1

    +
    2. +
  2. +

    Insert 2

    +
    3. +
+

WorstCase1

+
    +
  1. Splay 2 to the root
    2. +
+

WorstCase2

+
    +
  1. Insert 3
    2. +
+

WorstCase3

+
    +
  1. Splay 3 to the root
    2. +
+

WorstCase4

+
    +
  1. Insert 4
    2. +
+

WorstCase5

+
    +
  1. After inserting the rest of the values the tree will look like this:
    2. +
+

WorstCase6

+

If we keep insert number following the same sequence, that tree becomes inbalanced and have a height of n with n being the numbers of values inserted.
+After getting this tree, a find(1) operation for example will take O(n)

+

ZigZig rotation order: first grandparent

+

But thanks to the properties of the Splay Tree and the ZigZig rotations after the find(1) operation the tree becomes balanced again. This only happens if we respect the order of the ZigZig rotation, and the rotation to the grandparent happens first.

+

The sequence of ZigZigs rotations will look like follows:

+
    +
  1. Rotate 2 to 3
    2. +
+

ZigZig1

+
    +
  1. Rotate 1 to 2
    2. +
+

ZigZig2

+
    +
  1. Rotate 4 to 5
    2. +
+

ZigZig3

+
    +
  1. Rotate 1 to 4
    2. +
+

ZigZig4

+
    +
  1. Finally after splaying 1 to the root the tree will look like this:
    2. +
+

ZigZig5

+

Based on the example above, we can see why it's important to rotate first to the grandparent. We got a tree of height = 6, from an initial tree of height = 8. If the tree would had been taller, we would have achieved almost half of the initial height after the splaying operation.

+

ZigZig wrong rotation order

+

If the rotations would had been taking first the parent and not the grandparent we would have finished with the following, yet unbalanced tree, just inverting the side of the elements.

+

ZigZigWrong

+

See also

+

Splay Tree on Wikipedia

+

Splay Tree by University of California in Berkeley - CS 61B Lecture 34

+
+

Written for Swift Algorithm Club by Barbara Martina Rodeker

+
+ + diff --git a/Stack/index.html b/Stack/index.html new file mode 100644 index 000000000..3959b0f10 --- /dev/null +++ b/Stack/index.html @@ -0,0 +1,53 @@ + + + Stack + + + +

Stack

+
+

This topic has been tutorialized here

+
+

A stack is like an array but with limited functionality. You can only push to add a new element to the top of the stack, pop to remove the element from the top, and peek at the top element without popping it off.

+

Why would you want to do this? Well, in many algorithms you want to add objects to a temporary list at some point and then pull them off this list again at a later time. Often the order in which you add and remove these objects matters.

+

A stack gives you a LIFO or last-in first-out order. The element you pushed last is the first one to come off with the next pop. (A very similar data structure, the queue, is FIFO or first-in first-out.)

+

For example, let's push a number on the stack:

+
stack.push(10)
+

The stack is now [ 10 ]. Push the next number:

+
stack.push(3)
+

The stack is now [ 10, 3 ]. Push one more number:

+
stack.push(57)
+

The stack is now [ 10, 3, 57 ]. Let's pop the top number off the stack:

+
stack.pop()
+

This returns 57, because that was the most recent number we pushed. The stack is [ 10, 3 ] again.

+
stack.pop()
+

This returns 3, and so on. If the stack is empty, popping returns nil or in some implementations it gives an error message ("stack underflow").

+

A stack is easy to create in Swift. It's just a wrapper around an array that just lets you push, pop, and look at the top element of the stack:

+
public struct Stack<T> {
+  fileprivate var array = [T]()
+
+  public var isEmpty: Bool {
+    return array.isEmpty
+  }
+
+  public var count: Int {
+    return array.count
+  }
+
+  public mutating func push(_ element: T) {
+    array.append(element)
+  }
+
+  public mutating func pop() -> T? {
+    return array.popLast()
+  }
+
+  public var top: T? {
+    return array.last
+  }
+}
+

Notice that a push puts the new element at the end of the array, not the beginning. Inserting at the beginning of an array is expensive, an O(n) operation, because it requires all existing array elements to be shifted in memory. Adding at the end is O(1); it always takes the same amount of time, regardless of the size of the array.

+

Fun fact about stacks: Each time you call a function or a method, the CPU places the return address on a stack. When the function ends, the CPU uses that return address to jump back to the caller. That's why if you call too many functions -- for example in a recursive function that never ends -- you get a so-called "stack overflow" as the CPU stack has run out of space.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Strassen Matrix Multiplication/index.html b/Strassen Matrix Multiplication/index.html new file mode 100644 index 000000000..2c326976f --- /dev/null +++ b/Strassen Matrix Multiplication/index.html @@ -0,0 +1,369 @@ + + + Strassen Matrix Multiplication + + + +

Strassen Matrix Multiplication

+

Goal

+ +

What is Matrix Multiplication??

+
+

Note: If you are already familiar with Linear Algebra/Matrix Multiplication, feel free to skip this section

+
+

Before we begin, you may ask what is matrix multiplication? Great question! It is NOT multiplying two matricies term-by-term. Matrix multiplication is a mathematical operation that combines two matricies into a single one. Sounds like multiplying term-by-term huh? It's not... our lives would be much easier if it were. To see how matrix multiplcation works, let's look at an example.

+

Example: Matrix Multiplication

+
matrix A = |1 2|, matrix B = |5 6|
+           |3 4|             |7 8|
+	
+A * B = C
+	
+|1 2| * |5 6| = |1*5+2*7 1*6+2*8| = |19 22|
+|3 4|   |7 8|   |3*5+4*7 3*6+4*8|   |43 50|
+
+

What's going on here? To start, we're multiplying matricies A & B. Our new matrix, C's, elements [i, j] are determined by the dot product of the first matrix's ith row and the second matrix's jth column. See here for a refresher on the dot product.

+

So the upper left element [i=1, j=1] of our new matrix is a combination of A's 1st row and B's 1st column.

+
A's first row = [1, 2]
+B's first column = [5, 7]
+
+[1, 2] dot [5, 7] = [1*5 + 2*7] = [19] = C[1, 1]
+
+

Now let's try this for [i=1, j=2]. Because i=1 and j=2, this will represent the upper right element in our new matrix, C.

+
A's first row = [1, 2]
+B's second column = [6, 8]
+
+[1, 2] dot [6, 8] = [1*6 + 2*8] = [22] = C[1, 2]
+
+

If we do this for each row & column of A & B we'll get our result matrix C!

+

Here's a great graphic that visually shows you what's going on.

+

+

Source

+

Matix Multiplication Algorithm

+

So how do we implement matrix multiplication in an algoirthm? We'll start with the basic version and from there move on to Strassen's Algorithm.

+ +

Basic Version

+

Remember the method we used to solve matrix multiplication above? Let's try to implement that first! We first assert that the two matricies are the right size.

+

assert(A.columns == B.rows, "Two matricies can only be matrix mulitiplied if one has dimensions mxn & the other has dimensions nxp where m, n, p are in R")

+
+

NOTE: A's # of columns HAS to equal B's # of rows for matrix multiplication to work

+
+

Next, we loop over A's columns and B's rows. Because we know both A's columns & B's rows are the same length, we set that length equal to n.

+
for i in 0..<n {
+  for j in 0..<n {
+

Then, for each row in A and column in B, we take the dot product of the ith row in A with the jth column in B and set that result equal to the [i, j] element in C. Or C[i, j].

+
for k in 0..<n {
+  C[i, j] += A[i, k] * B[k, j]
+}
+

Finally, we return our new matrix C!

+

Here's the full implementation:

+
public func matrixMultiply(by B: Matrix<T>) -> Matrix<T> {
+  let A = self
+  assert(A.columns == B.rows, "Two matricies can only be matrix mulitiplied if one has dimensions mxn & the other has dimensions nxp where m, n, p are in R")
+  let n = A.columns
+  var C = Matrix<T>(rows: A.rows, columns: B.columns)
+    
+  for i in 0..<n {
+    for j in 0..<n {
+      for k in 0..<n {
+        C[i, j] += A[i, k] * B[k, j]
+      }
+    }
+  }
+    
+  return C
+}
+

This algorithm has a runtime of O(n^3). The O(n^3) comes from the three for loops. Two from the loop over the rows & columns and one from the dot product!

+

Now, O(n^3) is not very fast and a great question we should ask is can we do better? Indeed we can!

+

Strassens Algorithm

+

Volker Strassen first published his algorithm in 1969. It was the first algorithm to prove that the basic O(n^3) runtime was not optiomal.

+

The basic idea behind Strassen's algorithm is to split A & B into 8 submatricies and then recursively compute the submatricies of C. This strategy is called Divide and Conquer.

+
matrix A = |a b|, matrix B = |e f|
+           |c d|             |g h|
+
+

There will be 8 recursive calls:

+
    +
  1. a * e
    2. +
  2. b * g
    3. +
  3. a * f
    4. +
  4. b * h
    5. +
  5. c * e
    6. +
  6. d * g
    7. +
  7. c * f
    8. +
  8. d * h
    9. +
+

We then use these results to compute C's submatricies.

+
matrix C = |ae+bg af+bh|
+		   |ce+dg cf+dh| 
+
+

http://d1hyf4ir1gqw6c.cloudfront.net//wp-content/uploads/strassen_new.png

+

This step alone, however, doesn't help our runtime at all. Using the Master Theorem with T(n) = 8T(n/2) + O(n^2) we still get a runtime of O(n^3).

+

Strassen's insight was that we don't actually need 8 recursive calls to complete this process. We can finish the call with 7 recursive calls and a little bit of addition and subtraction.

+

Strassen's 7 calls are as follows:

+
    +
  1. a * (f - h)
    2. +
  2. (a + b) * h
    3. +
  3. (c + d) * e
    4. +
  4. d * (g - e)
    5. +
  5. (a + d) * (e + h)
    6. +
  6. (b - d) * (g + h)
    7. +
  7. (a - c) * (e + f)
    8. +
+

Now we can compute our new matrix C's new quardents!

+
matrix C = |p5+p4-p2+p6    p1+p2   |
+           |   p3+p4    p1+p5-p3-p7|    
+
+

A great reaction right now would be !!??!?!?!!?! How does this even work??

+

Let's prove it!

+

First Submatrix:

+
p5+p4-p2+p6 = (a+d)*(e+h) + d*(g-e) - (a+b)*h + (b-d)*(g+h)
+            = (ae+de+ah+dh) + (dg-de) - (ah+bh) + (bg-dg+bh-dh)
+            = ae+bg ✅
+
+

Exactly what we got the first time!

+

Now let's prove the others.

+

Second submatrix:

+
p1+p2 = a*(f-h) + (a+b)*h
+      = (af-ah) + (ah+bh)
+      = af+bh ✅
+
+

Third submatrix:

+
p3+p4 = (c+d)*e + d*(g-e)
+      = (ce+de) + (dg-de)
+      = ce+dg ✅
+
+

Fourth submatrix:

+
p1+p5-p3-p7 = a*(f-h) + (a+d)*(e+h) - (c+d)*e - (a-c)*(e+f)
+            = (af-ah) + (ae+de+ah+dh) -(ce+de) - (ae-ce+af-cf)
+            = cf+dh ✅
+
+

Great! The math checks out!

+

Here's the process in action.

+

+

Source

+

Implementation

+

Ok so now to the implementation. We'll start with the same first step from the basic implementation. We need to assert that A's # of columns are equal to B's number of rows.

+
assert(A.columns == B.rows, "Two matricies can only be matrix mulitiplied if one has dimensions mxn & the other has dimensions nxp where m, n, p are in R")
+
+

Now time for some prep work! We make each matrix a square and increase it's size to the next power of two. This ensures makes Strassen's Algorithm much easier to manage. We now only need to deal with square matricies that can be broken up an even number of times!

+
let n = max(A.rows, A.columns, B.rows, B.columns)
+let m = nextPowerOfTwo(after: n)
+    
+var APrep = Matrix(size: m)
+var BPrep = Matrix(size: m)
+   
+for i in A.rows {
+  for j in A.columns {
+    APrep[i, j] = A[i,j]
+  }
+}
+
+for i in B.rows {
+  for j in B.columns {
+    BPrep[i, j] = B[i, j]
+  }
+}
+

Finally, we recursively compute the matrix using Strassen's algorithm and the transform our new matrix C back to the correct dimensions!

+
let CPrep = APrep.strassenR(by: BPrep)
+var C = Matrix(rows: A.rows, columns: B.columns)
+    
+for i in 0..<A.rows {
+  for j in 0..<B.columns {
+    C[i,j] = CPrep[i,j]
+  }
+}
+

Recursively Computing the Matrix Multiplication

+

Next let's explore this recursive function strassenR.

+

We start by initializing 8 submatricies.

+
var a = Matrix(size: nBy2)
+var b = Matrix(size: nBy2)
+var c = Matrix(size: nBy2)
+var d = Matrix(size: nBy2)
+var e = Matrix(size: nBy2)
+var f = Matrix(size: nBy2)
+var g = Matrix(size: nBy2)
+var h = Matrix(size: nBy2)
+    
+for i in 0..<nBy2 {
+  for j in 0..<nBy2 {
+    a[i,j] = A[i,j]
+    b[i,j] = A[i, j+nBy2]
+    c[i,j] = A[i+nBy2, j]
+    d[i,j] = A[i+nBy2, j+nBy2]
+    e[i,j] = B[i,j]
+    f[i,j] = B[i, j+nBy2]
+    g[i,j] = B[i+nBy2, j]
+    h[i,j] = B[i+nBy2, j+nBy2]
+  }
+}
+

We next recursively compute the 7 matrix multiplications.

+
let p1 = a.strassenR(by: f-h)       // a * (f - h)
+let p2 = (a+b).strassenR(by: h)     // (a + b) * h
+let p3 = (c+d).strassenR(by: e)     // (c + d) * e
+let p4 = d.strassenR(by: g-e)       // d * (g - e)
+let p5 = (a+d).strassenR(by: e+h)   // (a + d) * (e + h)
+let p6 = (b-d).strassenR(by: g+h)   // (b - d) * (g + h)
+let p7 = (a-c).strassenR(by: e+f)   // (a - c) * (e + f)
+

Next, we compute the submatricies of C.

+
let c11 = p5 + p4 - p2 + p6         // p5 + p4 - p2 + p6
+let c12 = p1 + p2                   // p1 + p2
+let c21 = p3 + p4                   // p3 + p4
+let c22 = p1 + p5 - p3 - p7         // p1 + p5 - p3 - p7
+

And finally, we combine these submatricies into our new matrix C!

+
var C = Matrix(size: n)    
+for i in 0..<nBy2 {
+  for j in 0..<nBy2 {
+    C[i, j]           = c11[i,j]
+    C[i, j+nBy2]      = c12[i,j]
+    C[i+nBy2, j]      = c21[i,j]
+    C[i+nBy2, j+nBy2] = c22[i,j]
+  }
+}
+

As before, we can analyze the time completxity using the Master Theorem. T(n) = 7T(n/2) + O(n^2) which leads to O(n^log(7)) runtime. This comes out to approxiamtely O(n^2.8074) which is better than O(n^3)!

+

And that's Strassen's algorithm! Here's the full implementation:

+
// MARK: - Strassen Multiplication
+
+extension Matrix {
+  public func strassenMatrixMultiply(by B: Matrix<T>) -> Matrix<T> {
+    let A = self
+    assert(A.columns == B.rows, "Two matricies can only be matrix mulitiplied if one has dimensions mxn & the other has dimensions nxp where m, n, p are in R")
+    
+    let n = max(A.rows, A.columns, B.rows, B.columns)
+    let m = nextPowerOfTwo(after: n)
+    
+    var APrep = Matrix(size: m)
+    var BPrep = Matrix(size: m)
+    
+    A.forEach { (i, j) in
+      APrep[i,j] = A[i,j]
+    }
+    
+    B.forEach { (i, j) in
+      BPrep[i,j] = B[i,j]
+    }
+    
+    let CPrep = APrep.strassenR(by: BPrep)
+    var C = Matrix(rows: A.rows, columns: B.columns)
+    for i in 0..<A.rows {
+      for j in 0..<B.columns {
+        C[i,j] = CPrep[i,j]
+      }
+    }
+    
+    return C
+  }
+  
+  private func strassenR(by B: Matrix<T>) -> Matrix<T> {
+    let A = self
+    assert(A.isSquare && B.isSquare, "This function requires square matricies!")
+    guard A.rows > 1 && B.rows > 1 else { return A * B }
+    
+    let n    = A.rows
+    let nBy2 = n / 2
+    
+    /*
+    Assume submatricies are allocated as follows
+    
+     matrix A = |a b|,    matrix B = |e f|
+                |c d|                |g h|
+    */
+    
+    var a = Matrix(size: nBy2)
+    var b = Matrix(size: nBy2)
+    var c = Matrix(size: nBy2)
+    var d = Matrix(size: nBy2)
+    var e = Matrix(size: nBy2)
+    var f = Matrix(size: nBy2)
+    var g = Matrix(size: nBy2)
+    var h = Matrix(size: nBy2)
+    
+    for i in 0..<nBy2 {
+      for j in 0..<nBy2 {
+        a[i,j] = A[i,j]
+        b[i,j] = A[i, j+nBy2]
+        c[i,j] = A[i+nBy2, j]
+        d[i,j] = A[i+nBy2, j+nBy2]
+        e[i,j] = B[i,j]
+        f[i,j] = B[i, j+nBy2]
+        g[i,j] = B[i+nBy2, j]
+        h[i,j] = B[i+nBy2, j+nBy2]
+      }
+    }
+    
+    let p1 = a.strassenR(by: f-h)       // a * (f - h)
+    let p2 = (a+b).strassenR(by: h)     // (a + b) * h
+    let p3 = (c+d).strassenR(by: e)     // (c + d) * e
+    let p4 = d.strassenR(by: g-e)       // d * (g - e)
+    let p5 = (a+d).strassenR(by: e+h)   // (a + d) * (e + h)
+    let p6 = (b-d).strassenR(by: g+h)   // (b - d) * (g + h)
+    let p7 = (a-c).strassenR(by: e+f)   // (a - c) * (e + f)
+    
+    let c11 = p5 + p4 - p2 + p6         // p5 + p4 - p2 + p6
+    let c12 = p1 + p2                   // p1 + p2
+    let c21 = p3 + p4                   // p3 + p4
+    let c22 = p1 + p5 - p3 - p7         // p1 + p5 - p3 - p7
+    
+    var C = Matrix(size: n)
+    for i in 0..<nBy2 {
+      for j in 0..<nBy2 {
+        C[i, j]           = c11[i,j]
+        C[i, j+nBy2]      = c12[i,j]
+        C[i+nBy2, j]      = c21[i,j]
+        C[i+nBy2, j+nBy2] = c22[i,j]
+      }
+    }
+    
+    return C
+  }
+  
+  private func nextPowerOfTwo(after n: Int) -> Int {
+    return Int(pow(2, ceil(log2(Double(n)))))
+  }
+}
+

Appendix

+

Number Protocol

+

I use a number protocol to enable by Matrix to be generic.

+

The Number protocol ensures three things:

+
    +
  1. Everything that is a number can be multiplied
    2. +
  2. Everything that is a number can be added/subtracted
    3. +
  3. Everything that is a number has a zero value
    4. +
+

Extending Int, Float, and Double to conform to this protocol is now very straightforward. All you need to do is implement the static var zero!

+
public protocol Number: Multipliable, Addable {
+  static var zero: Self { get }
+}
+
+public protocol Addable {
+  static func +(lhs: Self, rhs: Self) -> Self
+  static func -(lhs: Self, rhs: Self) -> Self
+}
+
+public protocol Multipliable {
+  static func *(lhs: Self, rhs: Self) -> Self
+}
+

Dot Product

+

I extend Array to include a dot product function for when the Array's element conform to the Number protocol.

+
extension Array where Element: Number {
+  public func dot(_ b: Array<Element>) -> Element {
+    let a = self
+    assert(a.count == b.count, "Can only take the dot product of arrays of the same length!")
+    let c = a.indices.map{ a[$0] * b[$0] }
+    return c.reduce(Element.zero, { $0 + $1 })
+  }
+}
+

Resources

+ +

Written for Swift Algorithm Club by Richard Ash

+ + diff --git a/Ternary Search Tree/index.html b/Ternary Search Tree/index.html new file mode 100644 index 000000000..84d52e587 --- /dev/null +++ b/Ternary Search Tree/index.html @@ -0,0 +1,11 @@ + + + Ternary Search Tree + + + +

Ternary Search Tree

+

Data structure and simple test in playground has been implemented.
+Documentation and examples coming soon!! :)

+ + diff --git a/Threaded Binary Tree/index.html b/Threaded Binary Tree/index.html new file mode 100644 index 000000000..e8c53a585 --- /dev/null +++ b/Threaded Binary Tree/index.html @@ -0,0 +1,295 @@ + + + Threaded Binary Tree + + + +

Threaded Binary Tree

+

A threaded binary tree is a special kind of binary tree (a
+tree in which each node has at most two children) that maintains a few extra
+variables to allow cheap and fast in-order traversal of the tree. We will
+explore the general structure of threaded binary trees, as well as
+the Swift implementation of a fully functioning
+threaded binary tree.

+

If you don't know what a tree is or what it is for, then
+read this first.

+

In-order traversal

+

The main motivation behind using a threaded binary tree over a simpler and
+smaller standard binary tree is to increase the speed of an in-order traversal
+of the tree. An in-order traversal of a binary tree visits the nodes in the
+order in which they are stored, which matches the underlying ordering of a
+binary search tree. This means most threaded binary
+trees are also binary search trees. The idea is to visit all the left children
+of a node first, then visit the node itself, and then visit the right children
+last.

+

An in-order traversal of any binary tree generally goes as follows (using Swift
+syntax):

+
func traverse(n: Node?) {
+  if (n == nil) { return
+  } else {
+    traverse(n.left)
+    visit(n)
+    traverse(n.right)
+  }
+}
+

Where n is a a node in the tree (or nil), each node stores its children as
+left and right, and "visiting" a node can mean performing any desired
+action on it. We would call this function by passing to it the root of the
+tree we wish to traverse.

+

While simple and understandable, this algorithm uses stack space proportional
+to the height of the tree due to its recursive nature. If the tree has n
+nodes, this usage can range anywhere from O(log n) for a fairly balanced
+tree, to O(n) to a very unbalanced tree.

+

A threaded binary tree fixes this problem.

+
+

For more information about in-order traversals see here.

+
+

Predecessors and successors

+

An in-order traversal of a tree yields a linear ordering of the nodes. Thus
+each node has both a predecessor and a successor (except for the first
+and last nodes, which only have a successor or a predecessor respectively). In
+a threaded binary tree, each left child that would normally be nil instead
+stores the node's predecessor (if it exists), and each right child that would
+normally be nil instead stores the node's successor (if it exists). This is
+what separates threaded binary trees from standard binary trees.

+

There are two types of threaded binary trees: single threaded and double
+threaded:

+ +

Using a single or double threaded tree depends on what we want to accomplish.
+If we only need to traverse the tree in one direction (either forward or
+backward), then we use a single threaded tree. If we want to traverse in both
+directions, then we use a double threaded tree.

+

It is important to note that each node stores either its predecessor or its
+left child, and either its successor or its right child. The nodes do not
+need to keep track of both. For example, in a double threaded tree, if a node
+has a right child but no left child, it will track its predecessor in place of
+its left child.

+

Here is an example valid "full" threaded binary tree:

+

Full

+

While the following threaded binary tree is not "full," it is still valid. The
+structure of the tree does not matter as long as it follows the definition of a
+binary search tree:

+

Partial

+

The solid lines denote the links between parents and children, while the dotted
+lines denote the "threads." It is important to note how the children and
+thread edges interact with each other. Every node besides the root has one
+entering edge (from its parent), and two leaving edges: one to the left and one
+to the right. The left leaving edge goes to the node's left child if it
+exists, and to its in-order predecessor if it does not. The right leaving edge
+goes to the node's right child if it exists, and to its in-order successor if
+it does not. The exceptions are the left-most node and the right-most node,
+which do not have a predecessor or successor, respectively.

+

Representation

+

Before we go into detail about the methods of a threaded binary tree, we should
+first explain how the tree itself is represented. The core of this data
+structure is the ThreadedBinaryTree<T: Comparable> class. Each instance of
+this class represents a node with six member variables: value, parent,
+left, right, leftThread, and rightThread. Of all of these, only
+value is required. The other five are Swift optionals (they may be nil).

+ +

As we are storing both leftThread and rightThread, this is a double
+threaded tree. Now we are ready to go over some of the member functions in our
+ThreadedBinaryTree class.

+

Traversal algorithm

+

Let's start with the main reason we're using a threaded binary tree. It is now
+very easy to find the in-order predecessor and the in-order successor of any
+node in the tree. If the node has no left/right child, we can simply
+return the node's leftThread/rightThread. Otherwise, it is trivial to move
+down the tree and find the correct node.

+
  func predecessor() -> ThreadedBinaryTree<T>? {
+    if let left = left {
+      return left.maximum()
+    } else {
+      return leftThread
+    }
+  }
+
+  func successor() -> ThreadedBinaryTree<T>? {
+    if let right = right {
+      return right.minimum()
+    } else {
+      return rightThread
+    }
+  }
+
+

Note: maximum() and minimum() are methods of ThreadedBinaryTree which
+return the largest/smallest node in a given sub-tree. See
+the implementation for more detail.

+
+

Because these are ThreadedBinaryTree methods, we can call
+node.predecessor() or node.successor() to obtain the predecessor or
+successor of any node, provided that node is a ThreadedBinaryTree object.

+

Because predecessors and/or successors are tracked, an in-order traversal of a
+threaded binary tree is much more efficient than the recursive algorithm
+outlined above. We use these predecessor/successor attributes to great effect
+in this new algorithm for both forward and backward traversals:

+
    public func traverseInOrderForward(_ visit: (T) -> Void) {
+        var n: ThreadedBinaryTree
+        n = minimum()
+        while true {
+            visit(n.value)
+            if let successor = n.successor() {
+                n = successor
+            } else {
+                break
+            }
+        }
+    }
+
+    public func traverseInOrderBackward(_ visit: (T) -> Void) {
+        var n: ThreadedBinaryTree
+        n = maximum()
+        while true {
+            visit(n.value)
+            if let predecessor = n.predecessor() {
+                n = predecessor
+            } else {
+                break
+            }
+        }
+    }
+

Again, this a method of ThreadedBinaryTree, so we'd call it via
+node.traverseInorderForward(visitFunction). Note that we are able to specify
+a function that executes on each node as they are visited. This function can
+be anything you want, as long as it accepts T (the type of the values of the
+nodes of the tree) and has no return value.

+

Let's walk through a forward traversal of a tree by hand to get a better idea
+of how a computer would do it. For example, take this simple threaded tree:

+

Base

+

We start at the root of the tree, 9. Note that we don't visit(9) yet.
+From there we want to go to the minimum() node in the tree, which is 2 in
+this case. We then visit(2) and see that it has a rightThread, and thus
+we immediately know what its successor() is. We follow the thread to 5,
+which does not have any leaving threads. Therefore, after we visit(5), we go
+to the minimum() node in its right subtree, which is 7. We then
+visit(7) and see that it has a rightThread, which we follow to get back to
+9. Now we visit(9), and after noticing that it has no rightThread,
+we go to the minimum() node in its right subtree, which is 12. This
+node has a rightThread that leads to nil, which signals that we have
+completed the traversal! We visited the nodes in order 2, 5, 7, 9, 12,
+which intuitively makes sense, as that is their natural increasing order.

+

A backward traversal would be very similar, but you would replace right,
+rightThread, minimum(), and successor() with left, leftThread,
+maximum(), and predecessor().

+

Insertion and deletion

+

The quick in-order traversal that a threaded binary trees gives us comes at a
+small cost. Inserting/deleting nodes becomes more complicated, as we have to
+continuously manage the leftThread and rightThread variables. Rather than
+walking through some boring code, it is best to explain this with an example
+(although you can read through the implementation
+if you want to know the finer details). Please note that this requires
+knowledge of binary search trees, so make sure you have
+read this first.

+
+

Note: we do allow duplicate nodes in this implementation of a threaded binary
+tree. We break ties by defaulting insertion to the right.

+
+

Let's start with the same tree that we used for the above traversal example:

+

Base

+

Suppose we insert 10 into this tree. The resulting graph would look like
+this, with the changes highlighted in red:

+

Insert1

+

If you've done your homework and are familiar with binary search trees, the
+placement of this node should not surprise you. What's new is how we maintain
+the threads between nodes. So we know that we want to insert 10 as
+12's left child. The first thing we do is set 12's left child to
+10, and set 10's parent to 12. Because 10 is being inserted
+on the left, and 10 has no children of its own, we can safely set
+10's rightThread to its parent 12. What about 10's
+leftThread? Because we know that 10 < 12, and 10 is the only
+left child of 12, we can safely set 10's leftThread to 12's
+(now outdated) leftThread. Finally we set 12's leftThread = nil, as it
+now has a left child.

+

Let's now insert another node, 4, into the tree:

+

Insert2

+

While we are inserting 4 as a right child, it follows the exact same
+process as above, but mirrored (swap left and right). For the sake of
+completeness, we'll insert one final node, 15:

+

Insert3

+

Now that we have a fairly crowded tree, let's try removing some nodes.
+Compared to insertion, deletion is a little more complicated. Let's start with
+something simple, like removing 7, which has no children:

+

Remove1

+

Before we can just throw 7 away, we have to perform some clean-up. In this
+case, because 7 is a right child and has no children itself, we can
+simply set the rightThread of 7's parent(5) to 7's (now
+outdated) rightThread. Then we can just set 7's parent, left,
+right, leftThread, and rightThread to nil, effectively removing it from
+the tree. We also set the parent's rightChild to nil, which completes the deletion of this right child.

+

Let's try something a little harder. Say we remove 5 from the tree:

+

Remove2

+

This is a little trickier, as 5 has some children that we have to deal
+with. The core idea is to replace 5 with its first child, 2. To
+accomplish this, we of course set 2's parent to 9 and set 9's
+left child to 2. Note that 4's rightThread used to be 5, but
+we are removing 5, so it needs to change. It is now important to
+understand two important properties of threaded binary trees:

+
    +
  1. For the rightmost node m in the left subtree of any node n,
    +m's rightThread is n.
    2. +
  2. For the leftmost node m in the right subtree of any node n,
    +m's leftThread is n.
    3. +
+

Note how these properties held true before the removal of 5, as 4 was
+the rightmost node in 5's left subtree. In order to maintain this
+property, we must set 4's rightThread to 9, as 4 is now the
+rightmost node in 9's left subtree. To completely remove 5, all we
+now have to do is set 5's parent, left, right, leftThread, and
+rightThread to nil.

+

How about we do something crazy? What would happen if we tried to remove
+9, the root node? This is the resulting tree:

+

Remove3

+

Whenever we want to remove a node that has two children, we take a slightly
+different approach than the above examples. The basic idea is to replace the
+node that we want to remove with the leftmost node in its right subtree,
+which we call the replacement node.

+
+

Note: we could also replace the node with the rightmost node in its left
+subtree. Choosing left or right is mostly an arbitrary decision.

+
+

Once we find the replacement node, 10 in this case, we remove it from the
+tree using the algorithms outlined above. This ensures that the edges in the
+right subtree remain correct. From there it is easy to replace 9 with
+10, as we just have to update the edges leaving 10. Now all we have to
+do is fiddle with the threads in order to maintain the two properties outlined
+above. In this case, 12's leftThread is now 10. Node 9 is no
+longer needed, so we can finish the removal process by setting all of its
+variables to nil.

+

In order to illustrate how to remove a node that has only a right child,
+we'll remove one final node, 12 from the tree:

+

Remove4

+

The process to remove 12 is identical to the process we used to remove
+5, but mirrored. 5 had a left child, while 12 has a right
+child, but the core algorithm is the same.

+

And that's it! This was just a quick overview of how insertion and deletion
+work in threaded binary trees, but if you understood these examples, you should
+be able to insert or remove any node from any tree you want. More detail can
+of course be found in
+the implementation.

+

Miscellaneous methods

+

There are many other smaller operations that a threaded binary tree can do,
+such as searching() for a node in the tree, finding the depth() or
+height() of a node, etc. You can check
+the implementation for the full technical details.
+Many of these methods are inherent to binary search trees as well, so you can
+find further documentation here.

+

See also

+

Threaded Binary Tree on Wikipedia

+

Written for the Swift Algorithm Club by
+Jayson Tung
+Migrated to Swift 3 by Jaap Wijnen

+

Images made using www.draw.io

+ + diff --git a/Topological Sort/index.html b/Topological Sort/index.html new file mode 100644 index 000000000..ab1de1563 --- /dev/null +++ b/Topological Sort/index.html @@ -0,0 +1,112 @@ + + + Topological Sort + + + +

Topological Sort

+

Topological sort is an algorithm that orders a directed graph such that for each directed edge u→v, vertex u comes before vertex v.

+

In other words, a topological sort places the vertices of a directed acyclic graph on a line so that all directed edges go from left to right.

+

Consider the graph in the following example:

+

Example

+

This graph has two possible topological sorts:

+

Example

+

The topological orderings are S, V, W, T, X and S, W, V, T, X. Notice how the arrows all go from left to right.

+

The following is not a valid topological sort for this graph, since X and T cannot happen before V:

+

Example

+

Where is this used?

+

Let's consider that you want to learn all the algorithms and data structures from the Swift Algorithm Club. This might seem daunting at first but we can use topological sort to get things organized.

+

Since you're learning about topological sort, let's take this topic as an example. What else do you need to learn first before you can fully understand topological sort? Well, topological sort uses depth-first search as well as a stack. But before you can learn about the depth-first search algorithm, you need to know what a graph is, and it helps to know what a tree is. In turn, graphs and trees use the idea of linking objects together, so you may need to read up on that first. And so on...

+

If we were to represent these objectives in the form of a graph it would look as follows:

+

Example

+

If we consider each algorithm to be a vertex in the graph you can clearly see the dependencies between them. To learn something you might have to know something else first. This is exactly what topological sort is used for -- it will sort things out so that you know what to do first.

+

How does it work?

+

Step 1: Find all vertices that have in-degree of 0

+

The in-degree of a vertex is the number of edges pointing at that vertex. Vertices with no incoming edges have an in-degree of 0. These vertices are the starting points for the topological sort.

+

In the context of the previous example, these starting vertices represent algorithms and data structures that don't have any prerequisites; you don't need to learn anything else first, hence the sort starts with them.

+

Step 2: Traverse the graph with depth-first search

+

Depth-first search is an algorithm that starts traversing the graph from a certain vertex and explores as far as possible along each branch before backtracking. To find out more about depth-first search, please take a look at the detailed explanation.

+

We perform a depth-first search on each vertex with in-degree 0. This tells us which vertices are connected to each of these starting vertices.

+

Step 3: Remember all visited vertices

+

As we perform the depth-first search, we maintain a list of all the vertices that have been visited. This is to avoid visiting the same vertex twice.

+

Step 4: Put it all together

+

The last step of the sort is to combine the results of the different depth-first searches and put the vertices in a sorted list.

+

Example

+

Consider the following graph:

+

Graph Example

+

Step 1: The vertices with 0 in-degree are: 3, 7, 5. These are our starting vertices.

+

Step 2: Perform depth-first search for each starting vertex, without remembering vertices that have already been visited:

+
Vertex 3: 3, 10, 8, 9
+Vertex 7: 7, 11, 2, 8, 9
+Vertex 5: 5, 11, 2, 9, 10
+
+

Step 3: Filter out the vertices already visited in each previous search:

+
Vertex 3: 3, 10, 8, 9
+Vertex 7: 7, 11, 2
+Vertex 5: 5
+
+

Step 4: Combine the results of these three depth-first searches. The final sorted order is 5, 7, 11, 2, 3, 10, 8, 9. (Important: we need to add the results of each subsequent search to the front of the sorted list.)

+

The result of the topological sort looks like this:

+

Result of the sort

+
+

Note: This is not the only possible topological sort for this graph. For example, other valid solutions are 3, 7, 5, 10, 8, 11, 9, 2 and 3, 7, 5, 8, 11, 2, 9, 10. Any order where all the arrows are going from left to right will do.

+
+

The code

+

Here is how you could implement topological sort in Swift (see also TopologicalSort1.swift):

+
extension Graph {
+  public func topologicalSort() -> [Node] {
+    // 1
+    let startNodes = calculateInDegreeOfNodes().filter({ _, indegree in
+      return indegree == 0
+    }).map({ node, indegree in
+      return node
+    })
+    
+    // 2
+    var visited = [Node : Bool]()
+    for (node, _) in adjacencyLists {
+      visited[node] = false
+    }
+    
+    // 3
+    var result = [Node]()
+    for startNode in startNodes {
+      result = depthFirstSearch(startNode, visited: &visited) + result
+    }
+
+    // 4    
+    return result
+  }
+}
+

Some remarks:

+
    +
  1. +

    Find the in-degree of each vertex and put all the vertices with in-degree 0 in the startNodes array. In this graph implementation, vertices are called "nodes". Both terms are used interchangeably by people who write graph code.

    +
    2. +
  2. +

    The visited array keeps track of whether we've already seen a vertex during the depth-first search. Initially, we set all elements to false.

    +
    3. +
  3. +

    For each of the vertices in the startNodes array, perform a depth-first search. This returns an array of sorted Node objects. We prepend that array to our own result array.

    +
    4. +
  4. +

    The result array contains all the vertices in topologically sorted order.

    +
    5. +
+
+

Note: For a slightly different implementation of topological sort using depth-first search, see TopologicalSort3.swift. This uses a stack and does not require you to find all vertices with in-degree 0 first.

+
+

Kahn's algorithm

+

Even though depth-first search is the typical way to perform a topological sort, there is another algorithm that also does the job.

+
    +
  1. Find out what the in-degree is of every vertex.
    2. +
  2. Put all the vertices that have no predecessors in a new array called leaders. These vertices have in-degree 0 and therefore do not depend on any other vertices.
    3. +
  3. Go through this list of leaders and remove them one-by-one from the graph. We don't actually modify the graph, we just decrement the in-degree of the vertices they point to. That has the same effect.
    4. +
  4. Look at the (former) immediate neighbor vertices of each leader. If any of them now have an in-degree of 0, then they no longer have any predecessors themselves. We'll add those vertices to the leaders array too.
    5. +
  5. This repeats until there are no more vertices left to look at. At this point, the leaders array contains all the vertices in sorted order.
    6. +
+

This is an O(n + m) algorithm where n is the number of vertices and m is the number of edges. You can see the implementation in TopologicalSort2.swift.

+

Source: I first read about this alternative algorithm in the Algorithm Alley column in Dr. Dobb's Magazine from May 1993.

+

Written for Swift Algorithm Club by Ali Hafizji and Matthijs Hollemans

+ + diff --git a/Tree/index.html b/Tree/index.html new file mode 100644 index 000000000..50e2a766e --- /dev/null +++ b/Tree/index.html @@ -0,0 +1,137 @@ + + + Tree + + + +

Trees

+
+

This topic has been tutorialized here

+
+

A tree represents hierarchical relationships between objects. This is a tree:

+

A tree

+

A tree consists of nodes, and these nodes are linked to one another.

+

Nodes have links to their children and usually to their parent as well. The children are the nodes below a given node; the parent is the node above. A node always has just one parent but can have multiple children.

+

A tree

+

A node without a parent is the root node. A node without children is a leaf node.

+

The pointers in a tree do not form cycles. This is not a tree:

+

Not a tree

+

Such a structure is called a graph. A tree is really a very simple form of a graph. (In a similar vein, a linked list is a simple version of a tree. Think about it!)

+

This article talks about a general-purpose tree. That's a tree without any kind of restrictions on how many children each node may have, or on the order of the nodes in the tree.

+

Here's a basic implementation in Swift:

+
public class TreeNode<T> {
+  public var value: T
+
+  public weak var parent: TreeNode?
+  public var children = [TreeNode<T>]()
+
+  public init(value: T) {
+    self.value = value
+  }
+
+  public func addChild(_ node: TreeNode<T>) {
+    children.append(node)
+    node.parent = self
+  }
+}
+

This describes a single node from the tree. It has a value of generic type T, a reference to a parent node, and an array of child nodes.

+

It will be useful to add a description method so you can print the tree:

+
extension TreeNode: CustomStringConvertible {
+  public var description: String {
+    var s = "\(value)"
+    if !children.isEmpty {
+      s += " {" + children.map { $0.description }.joined(separator: ", ") + "}"
+    }
+    return s
+  }
+}
+

To see this in action in a playground:

+
let tree = TreeNode<String>(value: "beverages")
+
+let hotNode = TreeNode<String>(value: "hot")
+let coldNode = TreeNode<String>(value: "cold")
+
+let teaNode = TreeNode<String>(value: "tea")
+let coffeeNode = TreeNode<String>(value: "coffee")
+let chocolateNode = TreeNode<String>(value: "cocoa")
+
+let blackTeaNode = TreeNode<String>(value: "black")
+let greenTeaNode = TreeNode<String>(value: "green")
+let chaiTeaNode = TreeNode<String>(value: "chai")
+
+let sodaNode = TreeNode<String>(value: "soda")
+let milkNode = TreeNode<String>(value: "milk")
+
+let gingerAleNode = TreeNode<String>(value: "ginger ale")
+let bitterLemonNode = TreeNode<String>(value: "bitter lemon")
+
+tree.addChild(hotNode)
+tree.addChild(coldNode)
+
+hotNode.addChild(teaNode)
+hotNode.addChild(coffeeNode)
+hotNode.addChild(chocolateNode)
+
+coldNode.addChild(sodaNode)
+coldNode.addChild(milkNode)
+
+teaNode.addChild(blackTeaNode)
+teaNode.addChild(greenTeaNode)
+teaNode.addChild(chaiTeaNode)
+
+sodaNode.addChild(gingerAleNode)
+sodaNode.addChild(bitterLemonNode)
+

If you print out the value of tree, you'll get:

+
beverages {hot {tea {black, green, chai}, coffee, cocoa}, cold {soda {ginger ale, bitter lemon}, milk}}
+
+

That corresponds to the following structure:

+

Example tree

+

The beverages node is the root because it has no parent. The leaves are black, green, chai, coffee, cocoa, ginger ale, bitter lemon, milk because they don't have any child nodes.

+

For any node you can look at the parent property and work your way back up to the root:

+
teaNode.parent           // this is the "hot" node
+teaNode.parent!.parent   // this is the root
+

We often use the following definitions when talking about trees:

+ +

There are many different ways to construct trees. For example, sometimes you don't need to have a parent property at all. Or maybe you only need to give each node a maximum of two children -- such a tree is called a binary tree. A very common type of tree is the binary search tree (or BST), a stricter version of a binary tree where the nodes are ordered in a particular way to speed up searches.

+

The general purpose tree I've shown here is great for describing hierarchical data, but it really depends on your application what kind of extra functionality it needs to have.

+

Here's an example of how you could use the TreeNode class to determine if the tree contains a particular value. You first look at the node's own value property. If there's no match, then you look at all your children in turn. Of course, those children are also TreeNodes so they will repeat the same process recursively: first look at their own value and then at their children. And their children will also do the same thing again, and so on... Recursion and trees go hand-in-hand.

+

Here's the code:

+
extension TreeNode where T: Equatable {
+  func search(_ value: T) -> TreeNode? {
+    if value == self.value {
+      return self
+    }
+    for child in children {
+      if let found = child.search(value) {
+        return found
+      }
+    }
+    return nil
+  }
+}
+

And an example of how to use this:

+
tree.search("cocoa")    // returns the "cocoa" node
+tree.search("chai")     // returns the "chai" node
+tree.search("bubbly")   // nil
+

It's also possible to describe a tree using nothing more than an array. The indices in the array then create the links between the different nodes. For example, if we have:

+
0 = beverage		5 = cocoa		9  = green
+1 = hot			6 = soda		10 = chai
+2 = cold		7 = milk		11 = ginger ale
+3 = tea			8 = black		12 = bitter lemon
+4 = coffee				
+
+

Then we can describe the tree with the following array:

+
[ -1, 0, 0, 1, 1, 1, 2, 2, 3, 3, 3, 6, 6 ]
+
+

Each entry in the array is a pointer to its parent node. The item at index 3, tea, has the value 1 because its parent is hot. The root node points to -1 because it has no parent. You can only traverse such trees from a node back to the root but not the other way around.

+

By the way, sometimes you see algorithms using the term forest. Unsurprisingly, that is the name given to a collection of separate tree objects. For an example of this, see union-find.

+

Written for Swift Algorithm Club by Matthijs Hollemans

+ + diff --git a/Trie/index.html b/Trie/index.html new file mode 100644 index 000000000..923f898b2 --- /dev/null +++ b/Trie/index.html @@ -0,0 +1,144 @@ + + + Trie + + + +

Trie

+
+

This topic has been tutorialized here

+
+

What is a Trie?

+

A Trie, (also known as a prefix tree, or radix tree in some other implementations) is a special type of tree used to store associative data structures. A Trie for a dictionary might look like this:

+

A Trie

+

Storing the English language is a primary use case for a Trie. Each node in the Trie would represent a single character of a word. A series of nodes then make up a word.

+

Why a Trie?

+

Tries are very useful for certain situations. Here are some of the advantages:

+ +

Common Algorithms

+

Contains (or any general lookup method)

+

Trie structures are great for lookup operations. For Trie structures that model the English language, finding a particular word is a matter of a few pointer traversals:

+
func contains(word: String) -> Bool {
+  guard !word.isEmpty else { return false }
+
+  // 1
+  var currentNode = root
+  
+  // 2
+  var characters = Array(word.lowercased())
+  var currentIndex = 0
+ 
+  // 3
+  while currentIndex < characters.count, 
+    let child = currentNode.children[characters[currentIndex]] {
+
+    currentNode = child
+    currentIndex += 1
+  }
+
+  // 4
+  if currentIndex == characters.count && currentNode.isTerminating {
+    return true
+  } else {
+    return false
+  }
+}
+

The contains method is fairly straightforward:

+
    +
  1. Create a reference to the root. This reference will allow you to walk down a chain of nodes.
    2. +
  2. Keep track of the characters of the word you're trying to match.
    3. +
  3. Walk the pointer down the nodes.
    4. +
  4. isTerminating is a boolean flag for whether or not this node is the end of a word. If this if condition is satisfied, it means you are able to find the word in the trie.
    5. +
+

Insertion

+

Insertion into a Trie requires you to walk over the nodes until you either halt on a node that must be marked as terminating, or reach a point where you need to add extra nodes.

+
func insert(word: String) {
+  guard !word.isEmpty else {
+    return
+  }
+
+  // 1
+  var currentNode = root
+
+  // 2
+  for character in word.lowercased() {
+    // 3
+    if let childNode = currentNode.children[character] {
+      currentNode = childNode
+    } else {
+      currentNode.add(value: character)
+      currentNode = currentNode.children[character]!
+    }
+  }
+  // Word already present?
+  guard !currentNode.isTerminating else {
+    return
+  }
+
+  // 4
+  wordCount += 1
+  currentNode.isTerminating = true
+}
+
    +
  1. Once again, you create a reference to the root node. You'll move this reference down a chain of nodes.
    2. +
  2. Begin walking through your word letter by letter
    3. +
  3. Sometimes, the required node to insert already exists. That is the case for two words inside the Trie that shares letters (i.e "Apple", "App"). If a letter already exists, you'll reuse it, and simply traverse deeper down the chain. Otherwise, you'll create a new node representing the letter.
    4. +
  4. Once you get to the end, you mark isTerminating to true to mark that specific node as the end of a word.
    5. +
+

Removal

+

Removing keys from the trie is a little tricky, as there are a few more cases you'll need to take into account. Nodes in a Trie may be shared between different words. Consider the two words "Apple" and "App". Inside a Trie, the chain of nodes representing "App" is shared with "Apple".

+

If you'd like to remove "Apple", you'll need to take care to leave the "App" chain in tact.

+
func remove(word: String) {
+  guard !word.isEmpty else {
+    return
+  }
+
+  // 1
+  guard let terminalNode = findTerminalNodeOf(word: word) else {
+    return
+  }
+
+  // 2
+  if terminalNode.isLeaf {
+    deleteNodesForWordEndingWith(terminalNode: terminalNode)
+  } else {
+    terminalNode.isTerminating = false
+  }
+  wordCount -= 1
+}
+
    +
  1. findTerminalNodeOf traverses through the Trie to find the last node that represents the word. If it is unable to traverse through the chain of characters, it returns nil.
    2. +
  2. deleteNodesForWordEndingWith traverse backwords, deleting the nodes represented by the word.
    3. +
+

Time Complexity

+

Let n be the length of some value in the Trie.

+ +

Other Notable Operations

+ +

See also Wikipedia entry for Trie.

+

Written for the Swift Algorithm Club by Christian Encarnacion. Refactored by Kelvin Lau

+

Changes by Rick Zaccone

+
+ + diff --git a/Two-Sum Problem/index.html b/Two-Sum Problem/index.html new file mode 100644 index 000000000..ddaca8320 --- /dev/null +++ b/Two-Sum Problem/index.html @@ -0,0 +1,125 @@ + + + Two-Sum Problem + + + +

Two-Sum Problem

+

Given an array of integers and an integer target, return the indices of two numbers that add up to the target.

+

There are a variety of solutions to this problem (some better than others). The following solutions both run in O(n) time.

+

Solution 1

+

This solution looks at one number at a time, storing each number in the dictionary. It uses the number as the key and the number's index in the array as the value.

+

For each number n, we know the complementing number to sum up to the target is target - n. By looking up the complement in the dictionary, we'd know whether we've seen the complement before and what its index is.

+
func twoSum(_ nums: [Int], target: Int) -> (Int, Int)? {
+    var dict = [Int: Int]()
+    
+    // For every number n,
+    for (currentIndex, n) in nums.enumerated() {
+        // Find the complement to n that would sum up to the target.
+        let complement = target - n
+        
+        // Check if the complement is in the dictionary.
+        if let complementIndex = dict[complement] {
+            return (complementIndex, currentIndex)
+        }
+        
+        // Store n and its index into the dictionary.
+        dict[n] = currentIndex
+    }
+    
+    return nil
+}
+

The twoSum function takes two parameters: the numbers array and the target sum. It returns the two indicies of the pair of elements that sums up to the target, or nil if they can't be found.

+

Let's run through the algorithm to see how it works. Given the array:

+
[3, 2, 9, 8]
+

Let's find out if there exist two entries whose sum is 10.

+

Initially, our dictionary is empty. We begin looping through each element:

+ +

Is the complement 7 in the dictionary? No, so we add 3 and its index 0 to the dictionary.

+
[3: 0]
+ +

Is the complement 8 in the dictionary? No, so we add 2 and its index 1 to the dictionary.

+
[3: 0, 2: 1]
+ +

Is the complement 1 in the dictionary? No, so we add 9 and its index 2 to the dictionary.:

+
[3: 0, 2: 1, 9: 2]
+ +

Is the complement 2 in the dictionary? Yes! That means that we have found a pair of entries that sum to the target!

+

Therefore, the complementIndex = dict[2] = 1 and the currentIndex = 3. The tuple we return is (1, 3).

+

If the given array has multiple solutions, only the first solution is returned.

+

The running time of this algorithm is O(n) because it may look at every element in the array. It also requires O(n) additional storage space for the dictionary.

+

Solution 2

+

Note: This particular algorithm requires that the array is sorted, so if the array isn't sorted yet (usually it won't be), you need to sort it first. The time complexity of the algorithm itself is O(n) and, unlike the previous solution, it does not require extra storage. Of course, if you have to sort first, the total time complexity becomes O(n log n). Slightly worse but still quite acceptable.

+

Here is the code in Swift:

+
func twoSumProblem(_ a: [Int], k: Int) -> ((Int, Int))? {
+  var i = 0
+  var j = a.count - 1
+
+  while i < j {
+    let sum = a[i] + a[j]
+    if sum == k {
+      return (i, j)
+    } else if sum < k {
+      i += 1
+    } else {
+      j -= 1
+    }
+  }
+  return nil
+}
+

As in the first solution, the twoSumProblem() function takes as parameters the array a with the numbers and k, the sum we're looking for. If there are two numbers that add up to k, the function returns a tuple containing their array indices. If not, it returns nil. The main difference is that a is assumed to be sorted.

+

To test it, copy the code into a playground and add the following:

+
let a = [2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100]
+if let (i, j) = twoSumProblem(a, k: 33) {
+  a[i] + a[j]  // 33
+}
+

This returns the tuple (8, 10) because a[8] = 12 and a[10] = 21, and together they add up to 33.

+

So how does this algorithm work? It takes advantage of the array being sorted. That's true for many algorithms, by the way. If you first sort the data, it's often easier to perform your calculations.

+

In the example, the sorted array is:

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+
+

The algorithm uses the two variables i and j to point to the beginning and end of the array, respectively. Then it increments i and decrements j until the two meet. While it's doing this, it checks whether a[i] and a[j] add up to k.

+

Let's step through this. Initially, we have:

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+  i                                        j
+
+

The sum of these two is 2 + 100 = 102. That's obviously too much, since k = 33 in this example. There is no way that 100 will ever be part of the answer, so decrement j.

+

We have:

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+  i                                    j
+
+

The sum is 2 + 22 = 24. Now the sum is too small. We can safely conclude at this point that the number 2 will never be part of the answer. The largest number on the right is 22, so we at least need 11 on the left to make 33. Anything less than 11 is not going to cut it. (This is why we sorted the array!)

+

So, 2 is out and we increment i to look at the next small number.

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+     i                                 j
+
+

The sum is 3 + 22 = 25. Still too small, so increment i again.

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+        i                              j
+
+

In fact, we have to increment i a few more times, until we get to 12:

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+                           i           j
+
+

Now the sum is 12 + 22 = 34. It's too high, which means we need to decrement j. This gives:

+
[ 2, 3, 4, 4, 7, 8, 9, 10, 12, 14, 21, 22, 100 ]
+                           i       j
+
+

And finally, we have the answer: 12 + 21 = 33. Yay!

+

It's possible, of course, that there are no values for a[i] + a[j] that sum to k. In that case, eventually i and j will point at the same number. Then we can conclude that no answer exists and we return nil.

+

I'm quite enamored by this little algorithm. It shows that with some basic preprocessing on the input data -- sorting it from low to high -- you can turn a tricky problem into a very simple and beautiful algorithm.

+

Additional Reading

+ +

Written for Swift Algorithm Club by Matthijs Hollemans and Daniel Speiser updated to swift 4.2 by Farrukh Askari

+ + diff --git a/Union-Find/index.html b/Union-Find/index.html new file mode 100644 index 000000000..552ac8ce4 --- /dev/null +++ b/Union-Find/index.html @@ -0,0 +1,234 @@ + + + Union-Find + + + +

Union-Find

+

Union-Find is a data structure that can keep track of a set of elements partitioned into a number of disjoint (non-overlapping) subsets. It is also known as disjoint-set data structure.

+

What do we mean by this? For example, the Union-Find data structure could be keeping track of the following sets:

+
[ a, b, f, k ]
+[ e ]
+[ g, d, c ]
+[ i, j ]
+
+

These sets are disjoint because they have no members in common.

+

Union-Find supports three basic operations:

+
    +
  1. +

    Find(A): Determine which subset an element A is in. For example, find(d) would return the subset [ g, d, c ].

    +
    2. +
  2. +

    Union(A, B): Join two subsets that contain A and B into a single subset. For example, union(d, j) would combine [ g, d, c ] and [ i, j ] into the larger set [ g, d, c, i, j ].

    +
    3. +
  3. +

    AddSet(A): Add a new subset containing just that element A. For example, addSet(h) would add a new set [ h ].

    +
    4. +
+

The most common application of this data structure is keeping track of the connected components of an undirected graph. It is also used for implementing an efficient version of Kruskal's algorithm to find the minimum spanning tree of a graph.

+

Implementation

+

Union-Find can be implemented in many ways but we'll look at an efficient and easy to understand implementation: Weighted Quick Union.

+
+

PS: Multiple implementations of Union-Find has been included in playground.

+
+
public struct UnionFind<T: Hashable> {
+  private var index = [T: Int]()
+  private var parent = [Int]()
+  private var size = [Int]()
+}
+

Our Union-Find data structure is actually a forest where each subset is represented by a tree.

+

For our purposes we only need to keep track of the parent of each tree node, not the node's children. To do this we use the array parent so that parent[i] is the index of node i's parent.

+

Example: If parent looks like this,

+
parent [ 1, 1, 1, 0, 2, 0, 6, 6, 6 ]
+     i   0  1  2  3  4  5  6  7  8
+
+

then the tree structure looks like:

+
      1              6
+    /   \           / \
+  0       2        7   8
+ / \     /
+3   5   4
+
+

There are two trees in this forest, each of which corresponds to one set of elements. (Note: due to the limitations of ASCII art the trees are shown here as binary trees but that is not necessarily the case.)

+

We give each subset a unique number to identify it. That number is the index of the root node of that subset's tree. In the example, node 1 is the root of the first tree and 6 is the root of the second tree.

+

So in this example we have two subsets, the first with the label 1 and the second with the label 6. The Find operation actually returns the set's label, not its contents.

+

Note that the parent[] of a root node points to itself. So parent[1] = 1 and parent[6] = 6. That's how we can tell something is a root node.

+

Add set

+

Let's look at the implementation of these basic operations, starting with adding a new set.

+
public mutating func addSetWith(_ element: T) {
+  index[element] = parent.count  // 1
+  parent.append(parent.count)    // 2
+  size.append(1)                 // 3
+}
+

When you add a new element, this actually adds a new subset containing just that element.

+
    +
  1. +

    We save the index of the new element in the index dictionary. That lets us look up the element quickly later on.

    +
    2. +
  2. +

    Then we add that index to the parent array to build a new tree for this set. Here, parent[i] is pointing to itself because the tree that represents the new set contains only one node, which of course is the root of that tree.

    +
    3. +
  3. +

    size[i] is the count of nodes in the tree whose root is at index i. For the new set this is 1 because it only contains the one element. We'll be using the size array in the Union operation.

    +
    4. +
+

Find

+

Often we want to determine whether we already have a set that contains a given element. That's what the Find operation does. In our UnionFind data structure it is called setOf():

+
public mutating func setOf(_ element: T) -> Int? {
+  if let indexOfElement = index[element] {
+    return setByIndex(indexOfElement)
+  } else {
+    return nil
+  }
+}
+

This looks up the element's index in the index dictionary and then uses a helper method to find the set that this element belongs to:

+
private mutating func setByIndex(_ index: Int) -> Int {
+  if parent[index] == index {  // 1
+    return index
+  } else {
+    parent[index] = setByIndex(parent[index])  // 2
+    return parent[index]       // 3
+  }
+}
+

Because we're dealing with a tree structure, this is a recursive method.

+

Recall that each set is represented by a tree and that the index of the root node serves as the number that identifies the set. We're going to find the root node of the tree that the element we're searching for belongs to, and return its index.

+
    +
  1. +

    First, we check if the given index represents a root node (i.e. a node whose parent points back to the node itself). If so, we're done.

    +
    2. +
  2. +

    Otherwise we recursively call this method on the parent of the current node. And then we do a very important thing: we overwrite the parent of the current node with the index of root node, in effect reconnecting the node directly to the root of the tree. The next time we call this method, it will execute faster because the path to the root of the tree is now much shorter. Without that optimization, this method's complexity is O(n) but now in combination with the size optimization (covered in the Union section) it is almost O(1).

    +
    3. +
  3. +

    We return the index of the root node as the result.

    +
    4. +
+

Here's illustration of what I mean. Let's say the tree looks like this:

+

BeforeFind

+

We call setOf(4). To find the root node we have to first go to node 2 and then to node 7. (The indices of the elements are marked in red.)

+

During the call to setOf(4), the tree is reorganized to look like this:

+

AfterFind

+

Now if we need to call setOf(4) again, we no longer have to go through node 2 to get to the root. So as you use the Union-Find data structure, it optimizes itself. Pretty cool!

+

There is also a helper method to check that two elements are in the same set:

+
public mutating func inSameSet(_ firstElement: T, and secondElement: T) -> Bool {
+  if let firstSet = setOf(firstElement), let secondSet = setOf(secondElement) {
+    return firstSet == secondSet
+  } else {
+    return false
+  }
+}
+

Since this calls setOf() it also optimizes the tree.

+

Union (Weighted)

+

The final operation is Union, which combines two sets into one larger set.

+
    public mutating func unionSetsContaining(_ firstElement: T, and secondElement: T) {
+        if let firstSet = setOf(firstElement), let secondSet = setOf(secondElement) { // 1
+            if firstSet != secondSet {                // 2
+                if size[firstSet] < size[secondSet] { // 3
+                    parent[firstSet] = secondSet      // 4
+                    size[secondSet] += size[firstSet] // 5
+                } else {
+                    parent[secondSet] = firstSet
+                    size[firstSet] += size[secondSet]
+                }
+            }
+        }
+    }
+

Here is how it works:

+
    +
  1. +

    We find the sets that each element belongs to. Remember that this gives us two integers: the indices of the root nodes in the parent array.

    +
    2. +
  2. +

    Check that the sets are not equal because if they are it makes no sense to union them.

    +
    3. +
  3. +

    This is where the size optimization comes in (Weighting). We want to keep the trees as shallow as possible so we always attach the smaller tree to the root of the larger tree. To determine which is the smaller tree we compare trees by their sizes.

    +
    4. +
  4. +

    Here we attach the smaller tree to the root of the larger tree.

    +
    5. +
  5. +

    Update the size of larger tree because it just had a bunch of nodes added to it.

    +
    6. +
+

An illustration may help to better understand this. Let's say we have these two sets, each with its own tree:

+

BeforeUnion

+

Now we call unionSetsContaining(4, and: 3). The smaller tree is attached to the larger one:

+

AfterUnion

+

Note that, because we call setOf() at the start of the method, the larger tree was also optimized in the process -- node 3 now hangs directly off the root.

+

Union with optimizations also takes almost O(1) time.

+

Path Compression

+
private mutating func setByIndex(_ index: Int) -> Int {
+    if index != parent[index] {
+        // Updating parent index while looking up the index of parent.
+        parent[index] = setByIndex(parent[index])
+    }
+    return parent[index]
+}
+

Path Compression helps keep trees very flat, thus find operation could take ALMOST in O(1)

+

Complexity Summary

+
To process N objects
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Data StructureUnionFind
Quick FindN1
Quick UnionTree heightTree height
Weighted Quick UnionlgNlgN
Weighted Quick Union + Path Compressionvery close, but not O(1)very close, but not O(1)
+
To process M union commands on N objects
+ + + + + + + + + + + + + + + + + + + + + + + + + +
AlgorithmWorst-case time
Quick FindM N
Quick UnionM N
Weighted Quick UnionN + M lgN
Weighted Quick Union + Path Compression(M + N) lgN
+

See also

+

See the playground for more examples of how to use this handy data structure.

+

Union-Find at Wikipedia

+

Written for Swift Algorithm Club by Artur Antonov, modified by Yi Ding.

+ + diff --git a/What are Algorithms.html b/What are Algorithms.html new file mode 100644 index 000000000..8612ae7f5 --- /dev/null +++ b/What are Algorithms.html @@ -0,0 +1,22 @@ + + + What are Algorithms + + + +

What are algorithms and data structures?

+

An algorithm is a recipe for making the computer do something. If you know how to cook, you understand algorithms!

+

Here's a recipe for pancakes:

+
    +
  1. In a large bowl, sift together flour, baking powder, salt and sugar.
    2. +
  2. Pour in the milk, egg, and melted butter.
    3. +
  3. Mix until smooth.
    4. +
  4. Heat a frying pan over medium heat.
    5. +
  5. Scoop the batter into the pan, using approximately 1/4 cup for each pancake.
    6. +
  6. Brown the pancake on both sides.
    7. +
+

The recipe consists of a series of steps that you perform one after the other. An algorithm is just like that, except that it contains instructions for a computer to perform, not for a cook.

+

The ingredients -- flour, milk, eggs, butter -- are the data that the algorithm works on. The data goes into the algorithm in one form (raw, separate ingredients) and comes out in another (delicious pancakes!).

+

So what are the data structures? They are the containers that hold the data while the algorithm works on it. In the pancake recipe, the data structures are the bag that holds the flour, the mixing bowl where you combine everything, the frying pan that browns the pancake, and finally the plate used to serve the finished pancake.

+ + diff --git a/Why Algorithms.html b/Why Algorithms.html new file mode 100644 index 000000000..e841f1e1f --- /dev/null +++ b/Why Algorithms.html @@ -0,0 +1,27 @@ + + + Why Algorithms + + + +

Why learn algorithms and data structures?

+

If you've been coding for while you may wonder what the point is of learning about algorithms and data structures, especially if you don't have a formal computer science or engineering background.

+

After all, how often do you actually have to use a linked list or write your own sort routine when you're making apps? The answer is: almost never.

+

However...

+

Knowing a little bit about the strategies used by algorithms to solve tricky problems gives you ideas for improvements you can make to your own code.

+

Knowing more data structures than just the standard array and dictionary gives you a bigger collection of tools you can use to build your own apps.

+

It will make you a better developer! (And better developers make more $$$.)

+

Algorithms lets you build software you couldn't otherwise build

+

There have been apps that I've been unable to create in the past because I got stuck on fundamental issues.

+

Often it was a matter of speed: I just couldn't make the program go fast enough. Thinking back on this now, I had chosen the wrong algorithms for these problems. If I had known more about the difference between O(n) and O(n^2), then maybe I would have had better luck.

+

Naive brute-force solutions work fine for small amounts of data, but sometimes you need to deal with lots of data. And then you need smarter algorithms.

+

There were also times I wasn't able to solve my programming problems at all, not even slowly. I simply didn't know where to begin. Understanding a bit of algorithm theory gives you various tactics you can try.

+

Don't spend any time memorizing algorithms

+

That's not the point. Instead, try to understand how different algorithms approach different problems.

+

Learn about techniques such as divide-and-conquer, dynamic programming, greedy algorithms. See what makes one approach slow and another fast, and learn what the tradeoffs are.

+

The key thing here is to get insight in how we can make computers do things.

+

It's not as scary as it sounds

+

A lot of algorithm textbooks start with a bunch of math. Truth is, the math is useful but most of the time you won't need it. So don't let that scare you. If you can write code, you can also understand all these fancy algorithms and data structures.

+

Trust me, algorithms are fun. :-)

+ + diff --git a/Z-Algorithm/index.html b/Z-Algorithm/index.html new file mode 100644 index 000000000..844f07e3f --- /dev/null +++ b/Z-Algorithm/index.html @@ -0,0 +1,151 @@ + + + Z-Algorithm + + + +

Z-Algorithm String Search

+

Goal: Write a simple linear-time string matching algorithm in Swift that returns the indexes of all the occurrencies of a given pattern.

+

In other words, we want to implement an indexesOf(pattern: String) extension on String that returns an array [Int] of integers, representing all occurrences' indexes of the search pattern, or nil if the pattern could not be found inside the string.

+

For example:

+
let str = "Hello, playground!"
+str.indexesOf(pattern: "ground")   // Output: [11]
+
+let traffic = "🚗🚙🚌🚕🚑🚐🚗🚒🚚🚎🚛🚐🏎🚜🚗🏍🚒🚲🚕🚓🚌🚑"
+traffic.indexesOf(pattern: "🚑") // Output: [4, 21]
+

Many string search algorithms use a pre-processing function to compute a table that will be used in successive stage. This table can save some time during the pattern search stage because it allows to avoid un-needed characters comparisons. The Z-Algorithm is one of these functions. It borns as a pattern pre-processing function (this is its role in the Knuth-Morris-Pratt algorithm and others) but, just like we will show here, it can be used also as a single string search algorithm.

+

Z-Algorithm as pattern pre-processor

+

As we said, the Z-Algorithm is foremost an algorithm that process a pattern in order to calculate a skip-comparisons-table.
+The computation of the Z-Algorithm over a pattern P produces an array (called Z in the literature) of integers in which each element, call it Z[i], represents the length of the longest substring of P that starts at i and matches a prefix of P. In simpler words, Z[i] records the longest prefix of P[i...|P|] that matches a prefix of P. As an example, let's consider P = "ffgtrhghhffgtggfredg". We have that Z[5] = 0 (f...h...), Z[9] = 4 (ffgtr...ffgtg...) and Z[15] = 1 (ff...fr...).

+

But how do we compute Z? Before we describe the algorithm we must indroduce the concept of Z-box. A Z-box is a pair (left, right) used during the computation that records the substring of maximal length that occurs also as a prefix of P. The two indices left and right represent, respectively, the left-end index and the right-end index of this substring.
+The definition of the Z-Algorithm is inductive and it computes the elements of the array for every position k in the pattern, starting from k = 1. The following values (Z[k + 1], Z[k + 2], ...) are computed after Z[k]. The idea behind the algorithm is that previously computed values can speed up the calculus of Z[k + 1], avoiding some character comparisons that were already done before. Consider this example: suppose we are at iteration k = 100, so we are analyzing position 100 of the pattern. All the values between Z[1] and Z[99] were correctly computed and left = 70 and right = 120. This means that there is a substring of length 51 starting at position 70 and ending at position 120 that matches the prefix of the pattern/string we are considering. Reasoning on it a little bit we can say that the substring of length 21 starting at position 100 matches the substring of length 21 starting at position 30 of the pattern (because we are inside a substring that matches a prefix of the pattern). So we can use Z[30] to compute Z[100] without additional character comparisons.
+This a simple description of the idea that is behind this algorithm. There are a few cases to manage when the use of pre-computed values cannot be directly applied and some comparisons are to be made.

+

Here is the code of the function that computes the Z-array:

+
func ZetaAlgorithm(ptrn: String) -> [Int]? {
+    let pattern = Array(ptrn)
+    let patternLength: Int = pattern.count
+
+    guard patternLength > 0 else {
+        return nil
+    }
+
+    var zeta: [Int] = [Int](repeating: 0, count: patternLength)
+
+    var left: Int = 0
+    var right: Int = 0
+    var k_1: Int = 0
+    var betaLength: Int = 0
+    var textIndex: Int = 0
+    var patternIndex: Int = 0
+
+    for k in 1 ..< patternLength {
+        if k > right {  // Outside a Z-box: compare the characters until mismatch
+            patternIndex = 0
+
+            while k + patternIndex < patternLength  &&
+                pattern[k + patternIndex] == pattern[patternIndex] {
+                patternIndex = patternIndex + 1
+            }
+
+            zeta[k] = patternIndex
+
+            if zeta[k] > 0 {
+                left = k
+                right = k + zeta[k] - 1
+            }
+        } else {  // Inside a Z-box
+            k_1 = k - left + 1
+            betaLength = right - k + 1
+
+            if zeta[k_1 - 1] < betaLength { // Entirely inside a Z-box: we can use the values computed before
+                zeta[k] = zeta[k_1 - 1]
+            } else if zeta[k_1 - 1] >= betaLength { // Not entirely inside a Z-box: we must proceed with comparisons too
+                textIndex = betaLength
+                patternIndex = right + 1
+
+                while patternIndex < patternLength && pattern[textIndex] == pattern[patternIndex] {
+                    textIndex = textIndex + 1
+                    patternIndex = patternIndex + 1
+                }
+
+                zeta[k] = patternIndex - k
+                left = k
+                right = patternIndex - 1
+            }
+        }
+    }
+    return zeta
+}
+

Let's make an example reasoning with the code above. Let's consider the string P = “abababbb". The algorithm begins with k = 1, left = right = 0. So, no Z-box is "active" and thus, because k > right we start with the character comparisons beetwen P[1] and P[0].

+
   01234567
+k:  x
+   abababbb
+   x
+Z: 00000000
+left:  0
+right: 0
+
+

We have a mismatch at the first comparison and so the substring starting at P[1] does not match a prefix of P. So, we put Z[1] = 0 and let left and right untouched. We begin another iteration with k = 2, we have 2 > 0 and again we start comparing characters P[2] with P[0]. This time the characters match and so we continue the comparisons until a mismatch occurs. It happens at position 6. The characters matched are 4, so we put Z[2] = 4 and set left = k = 2 and right = k + Z[k] - 1 = 5. We have our first Z-box that is the substring "abab" (notice that it matches a prefix of P) starting at position left = 2.

+
   01234567
+k:   x
+   abababbb
+   x
+Z: 00400000
+left:  2
+right: 5
+
+

We then proceed with k = 3. We have 3 <= 5. We are inside the Z-box previously found and inside a prefix of P. So we can look for a position that has a previously computed value. We calculate k_1 = k - left = 1 that is the index of the prefix's character equal to P[k]. We check Z[1] = 0 and 0 < (right - k + 1 = 3) and we find that we are exactly inside the Z-box. We can use the previously computed value, so we put Z[3] = Z[1] = 0, left and right remain unchanged.
+At iteration k = 4 we initially execute the else branch of the outer if. Then in the inner if we have that k_1 = 2 and (Z[2] = 4) >= 5 - 4 + 1. So, the substring P[k...r] matches for right - k + 1 = 2 chars the prefix of P but it could not for the following characters. We must then compare the characters starting at r + 1 = 6 with those starting at right - k + 1 = 2. We have P[6] != P[2] and so we have to set Z[k] = 6 - 4 = 2, left = 4 and right = 5.

+
   01234567
+k:     x
+   abababbb
+   x
+Z: 00402000
+left:  4
+right: 5
+
+

With iteration k = 5 we have k <= right and then (Z[k_1] = 0) < (right - k + 1 = 1) and so we set z[k] = 0. In iteration 6 and 7 we execute the first branch of the outer if but we only have mismatches, so the algorithms terminates returning the Z-array as Z = [0, 0, 4, 0, 2, 0, 0, 0].

+

The Z-Algorithm runs in linear time. More specifically, the Z-Algorithm for a string P of size n has a running time of O(n).

+

The implementation of Z-Algorithm as string pre-processor is contained in the ZAlgorithm.swift file.

+

Z-Algorithm as string search algorithm

+

The Z-Algorithm discussed above leads to the simplest linear-time string matching algorithm. To obtain it, we have to simply concatenate the pattern P and text T in a string S = P$T where $ is a character that does not appear neither in P nor T. Then we run the algorithm on S obtaining the Z-array. All we have to do now is scan the Z-array looking for elements equal to n (which is the pattern length). When we find such value we can report an occurrence.

+
extension String {
+
+    func indexesOf(pattern: String) -> [Int]? {
+        let patternLength: Int = pattern.count
+        /* Let's calculate the Z-Algorithm on the concatenation of pattern and text */
+        let zeta = ZetaAlgorithm(ptrn: pattern + "💲" + self)
+
+        guard zeta != nil else {
+            return nil
+        }
+
+        var indexes: [Int] = [Int]()
+
+        /* Scan the zeta array to find matched patterns */
+        for i in 0 ..< zeta!.count {
+            if zeta![i] == patternLength {
+                indexes.append(i - patternLength - 1)
+            }
+        }
+
+        guard !indexes.isEmpty else {
+            return nil
+        }
+
+        return indexes
+    }
+}
+

Let's make an example. Let P = “CATA“ and T = "GAGAACATACATGACCAT" be the pattern and the text. Let's concatenate them with the character $. We have the string S = "CATA$GAGAACATACATGACCAT". After computing the Z-Algorithm on S we obtain:

+
            1         2
+  01234567890123456789012
+  CATA$GAGAACATACATGACCAT
+Z 00000000004000300001300
+            ^
+
+

We scan the Z-array and at position 10 we find Z[10] = 4 = n. So we can report a match occuring at text position 10 - n - 1 = 5.

+

As said before, the complexity of this algorithm is linear. Defining n and m as pattern and text lengths, the final complexity we obtain is O(n + m + 1) = O(n + m).

+

Credits: This code is based on the handbook "Algorithm on String, Trees and Sequences: Computer Science and Computational Biology" by Dan Gusfield, Cambridge University Press, 1997.

+

Written for Swift Algorithm Club by Matteo Dunnhofer

+ + diff --git a/github-light.css b/github-light.css new file mode 100644 index 000000000..63972837a --- /dev/null +++ b/github-light.css @@ -0,0 +1,175 @@ +/*! + * GitHub Light v0.5.0 + * Copyright (c) 2012 - 2017 GitHub, Inc. + * Licensed under MIT (https://github.com/primer/github-syntax-theme-generator/blob/master/LICENSE) + */ + +.pl-c /* comment, punctuation.definition.comment, string.comment */ { + color: #6a737d; +} + +.pl-c1 /* constant, entity.name.constant, variable.other.constant, variable.language, support, meta.property-name, support.constant, support.variable, meta.module-reference, markup.raw, meta.diff.header, meta.output */, +.pl-s .pl-v /* string variable */ { + color: #005cc5; +} + +.pl-e /* entity */, +.pl-en /* entity.name */ { + color: #6f42c1; +} + +.pl-smi /* variable.parameter.function, storage.modifier.package, storage.modifier.import, storage.type.java, variable.other */, +.pl-s .pl-s1 /* string source */ { + color: #24292e; +} + +.pl-ent /* entity.name.tag, markup.quote */ { + color: #22863a; +} + +.pl-k /* keyword, storage, storage.type */ { + color: #d73a49; +} + +.pl-s /* string */, +.pl-pds /* punctuation.definition.string, source.regexp, string.regexp.character-class */, +.pl-s .pl-pse .pl-s1 /* string punctuation.section.embedded source */, +.pl-sr /* string.regexp */, +.pl-sr .pl-cce /* string.regexp constant.character.escape */, +.pl-sr .pl-sre /* string.regexp source.ruby.embedded */, +.pl-sr .pl-sra /* string.regexp string.regexp.arbitrary-repitition */ { + color: #032f62; +} + +.pl-v /* variable */, +.pl-smw /* sublimelinter.mark.warning */ { + color: #e36209; +} + +.pl-bu /* invalid.broken, invalid.deprecated, invalid.unimplemented, message.error, brackethighlighter.unmatched, sublimelinter.mark.error */ { + color: #b31d28; +} + +.pl-ii /* invalid.illegal */ { + color: #fafbfc; + background-color: #b31d28; +} + +.pl-c2 /* carriage-return */ { + color: #fafbfc; + background-color: #d73a49; +} + +.pl-c2::before /* carriage-return */ { + content: "^M"; +} + +.pl-sr .pl-cce /* string.regexp constant.character.escape */ { + font-weight: bold; + color: #22863a; +} + +.pl-ml /* markup.list */ { + color: #735c0f; +} + +.pl-mh /* markup.heading */, +.pl-mh .pl-en /* markup.heading entity.name */, +.pl-ms /* meta.separator */ { + font-weight: bold; + color: #005cc5; +} + +.pl-mi /* markup.italic */ { + font-style: italic; + color: #24292e; +} + +.pl-mb /* markup.bold */ { + font-weight: bold; + color: #24292e; +} + +.pl-md /* markup.deleted, meta.diff.header.from-file, punctuation.definition.deleted */ { + color: #b31d28; + background-color: #ffeef0; +} + +.pl-mi1 /* markup.inserted, meta.diff.header.to-file, punctuation.definition.inserted */ { + color: #22863a; + background-color: #f0fff4; +} + +.pl-mc /* markup.changed, punctuation.definition.changed */ { + color: #e36209; + background-color: #ffebda; +} + +.pl-mi2 /* markup.ignored, markup.untracked */ { + color: #f6f8fa; + background-color: #005cc5; +} + +.pl-mdr /* meta.diff.range */ { + font-weight: bold; + color: #6f42c1; +} + +.pl-ba /* brackethighlighter.tag, brackethighlighter.curly, brackethighlighter.round, brackethighlighter.square, brackethighlighter.angle, brackethighlighter.quote */ { + color: #586069; +} + +.pl-sg /* sublimelinter.gutter-mark */ { + color: #959da5; +} + +.pl-corl /* constant.other.reference.link, string.other.link */ { + text-decoration: underline; + color: #032f62; +} + +#container { + margin: 0 auto; + width: 75%; + min-width: 768px; + max-width: 896px; + position: relative; +} + +body { + font-size: 18px; +} + +code { + padding: 0.2em; + margin: 0; + font-size: 85%; + background-color: #f6f8fa; + line-height: 1.45; + border-radius: 3px +} + +pre code { + padding: 0px; + background-color: transparent; +} + +.highlight { + margin: 0px; + padding: 0px 16px; + font-size: 85%; + line-height: 1.45; + overflow: auto; + background-color: #f6f8fa; + border-radius: 3px; +} + +@media (max-width: 768px) { + #container { + position: absolute; + margin: 0; + width: 100%; + height: 100%; + min-width: 100%; + } +} diff --git a/index.html b/index.html new file mode 100644 index 000000000..a0766ca78 --- /dev/null +++ b/index.html @@ -0,0 +1,215 @@ + + + Swift Algorithm Club + + + +

Swift Algorithm Club

+

Welcome to the Swift Algorithm Club!

+

Here you'll find implementations of popular algorithms and data structures in everyone's favorite new language Swift, with detailed explanations of how they work.

+

If you're a computer science student who needs to learn this stuff for exams -- or if you're a self-taught programmer who wants to brush up on the theory behind your craft -- you've come to the right place!

+

The goal of this project is to explain how algorithms work. The focus is on clarity and readability of the code, not on making a reusable library that you can drop into your own projects. That said, most of the code should be ready for production use but you may need to tweak it to fit into your own codebase.

+

Code is compatible with Xcode 10 and Swift 4.2. We'll keep this updated with the latest version of Swift. If you're interested in a GitHub pages version of the repo, check out this.

+

😍 Suggestions and contributions are welcome! 😍

+

Important links

+

What are algorithms and data structures? Pancakes!

+

Why learn algorithms? Worried this isn't your cup of tea? Then read this.

+

Big-O notation. We often say things like, "This algorithm is O(n)." If you don't know what that means, read this first.

+

Algorithm design techniques. How do you create your own algorithms?

+

How to contribute. Report an issue to leave feedback, or submit a pull request.

+

Where to start?

+

If you're new to algorithms and data structures, here are a few good ones to start out with:

+ +

The algorithms

+

Searching

+ +

String Search

+ +

Sorting

+

It's fun to see how sorting algorithms work, but in practice you'll almost never have to provide your own sorting routines. Swift's own sort() is more than up to the job. But if you're curious, read on...

+

Basic sorts:

+ +

Fast sorts:

+ +

Hybrid sorts:

+ +

Special-purpose sorts:

+ +

Bad sorting algorithms (don't use these!):

+ +

Compression

+ +

Miscellaneous

+ +

Mathematics

+ +

Machine learning

+ +

Data structures

+

The choice of data structure for a particular task depends on a few things.

+

First, there is the shape of your data and the kinds of operations that you'll need to perform on it. If you want to look up objects by a key you need some kind of dictionary; if your data is hierarchical in nature you want a tree structure of some sort; if your data is sequential you want a stack or queue.

+

Second, it matters what particular operations you'll be performing most, as certain data structures are optimized for certain actions. For example, if you often need to find the most important object in a collection, then a heap or priority queue is more optimal than a plain array.

+

Most of the time using just the built-in Array, Dictionary, and Set types is sufficient, but sometimes you may want something more fancy...

+

Variations on arrays

+ +

Queues

+ +

Lists

+ +

Trees

+ +

Hashing

+ +

Sets

+ +

Graphs

+ +

Puzzles

+

A lot of software developer interview questions consist of algorithmic puzzles. Here is a small selection of fun ones. For more puzzles (with answers), see here and here.

+ +

Learn more!

+

Like what you see? Check out Data Structures & Algorithms in Swift, the official book by the Swift Algorithm Club team!

+

Data Structures & Algorithms in Swift Book

+

You’ll start with the fundamental structures of linked lists, queues and stacks, and see how to implement them in a highly Swift-like way. Move on to working with various types of trees, including general purpose trees, binary trees, AVL trees, binary search trees, and tries.

+

Go beyond bubble and insertion sort with better-performing algorithms, including mergesort, radix sort, heap sort, and quicksort. Learn how to construct directed, non-directed and weighted graphs to represent many real-world models, and traverse graphs and trees efficiently with breadth-first, depth-first, Dijkstra’s and Prim’s algorithms to solve problems such as finding the shortest path or lowest cost in a network.

+

By the end of this book, you’ll have hands-on experience solving common issues with data structures and algorithms — and you’ll be well on your way to developing your own efficient and useful implementations!

+

You can find the book on the raywenderlich.com store.

+

Credits

+

The Swift Algorithm Club was originally created by Matthijs Hollemans.

+

It is now maintained by Vincent Ngo, Kelvin Lau, and Richard Ash.

+

The Swift Algorithm Club is a collaborative effort from the most algorithmic members of the raywenderlich.com community. We're always looking for help - why not join the club? :]

+

License

+

All content is licensed under the terms of the MIT open source license.

+

By posting here, or by submitting any pull request through this forum, you agree that all content you submit or create, both code and text, is subject to this license. Razeware, LLC, and others will have all the rights described in the license regarding this content. The precise terms of this license may be found here.

+

Build Status

+ +