Dynamic programming is a powerful tool for solving complex problems. And memoization is a routine trick that is applied in dynamic programming. As two important concepts, some new learners find it confusing. At algo.monster, let’s compare these two techniques to tell the difference. You’ll have a better understanding of the DP programming language.
General explanations of these two techniques
First of all, let’s figure what they are before we come to further discussion.
What is dynamic programming?
Dynamic Programming is an algorithmic way to solve complex problems. This is achieved by recursively breaking down the original problem into smaller subproblems. Then , the optimal solution to each subproblem determines the best solution to the given problem.
In other words, DP is solving problems recursively. An optimal solution will be found by combining the outcomes of similar smaller overlapping subproblems.
Two properties that a DP problem must contain: optimal substructures and overlapping subproblems.
It means to the process of solving smaller subproblems is to find the final result for the target problem. When all subproblems are solved, combine the smaller optimal solutions, then you’ll get the overall optimal solution.
The original complex problem is made up of a series of similar subproblems. These subproblems are interdependent. In other words, they overlap. That is to say, the same subproblems will appear more than once.
Why is DP called dynamic programming?
The name was chosen by its creator, Richard Bellman. The main idea of using dynamic is to capture the time-varying aspect of the problems. Besides, he thought it sounded impressive. If you want to know the detailed story about it, check this book out: Eye of the Hurricane: An Autobiography (1984). It is Bellman’s autobiography. In this book, he gave an explanation of the reasoning behind the term “dynamic programming”.
Today, you would define “dynamic” as “moving from smaller subproblems to larger ones”. Programming refers to the process of finding the optimal program. This is similar to “linear programming”. Yet, we sometimes refer to it as software programming.
Two approaches to DP
Top-down: The problem-solving process is moving from the top to the bottom. This method caches outcomes of subproblems. Then it can reuse the caches when encountering the same subproblems again. With the caching method-memoization, it avoids unnecessary recalculation of any kind.
Bottom-up: Bottom-up is a process from bottom to top, through populating into an n-dimensional table. So, that’s why people call it the tabulation approach. This method begins from the bottom and completes at the top. By solving related subproblems, it prevents recursion in this approach.
What is memoization?
The right word is memoization, not memorization. In fact, it is from memorize or memoize.
Memoization refers to the ability to memorize or store the outcomes of a computation. It is a top-down approach for tackling problems. Normally, it creates a memory for future reference. As a result, the previous outcomes can be reused to increase time efficiency next time.
Why do some think DP and memoization are the same?
Some people believe that dynamic Programming is merely another name for memoization or any other tricks utilizing memoization. The most important aspect of DP is the remarkable speedup provided by memoization.
Actually, though not accurate, equating DP to memoization technique applied recursively algorithms are acceptable. Recursive algorithms such as computation of possible routes from the left-bottom corner to the top-right corner of a rectangle grid, or Fibonacci sequence calculation.
When the memoization technique shows up in dynamic programming problems, this auxiliary routine trick optimizes the performance of DP. The frequent existence of this technique makes people think that they’re the same.
What’s the difference between dynamic programming and memoization?
To illustrate the difference between memoization and DP, let me give you a simple example of DP. This is the maximum sub-array problem you can solve by using Kadane’s algorithm. You will realize that, there’s no memoization in this algorithm.
As a child, I wondered how to find the maximum sum in a contiguous sub-array for a given array. Combining adjacent positive numbers with adjacent negative numbers was my first thought. By doing that, it would simplify the input. Next, I combined neighboring numbers if their sum was positive or negative. The uncertainty started to mess up with my approach from all sides. I was happy when I discovered the Kadane algorithm. This algorithm is so simple. Thanks to Kadane, every solution will always contain the last element. You can only fully understand the essence of dynamic programming if you can see the power in such a simple observation.
Dynamic programming is a solution approach that requires you to find similar simpler subproblems to compute bigger subproblems. And this process often contains memoization and recurrence relations.
Memoization is a tool that prevents calculating the same subproblems that repeat more than once. Thus, you could regard it as a special way of memorizing values by caching the results.
Conclusion and suggestion
Memoization is never just another name of dynamic programming. They are different problem-solving approaches that are similar.
- In a DP algorithm, you often witness memoization and recurrence. This strategy moves from similar smaller subproblems to larger problems to find the overall optimal solution.
- Memoization is similar to the technique of caching. It stores the values of a function for future reference. For that reason, no unnecessary repetition will occur anymore.
- DP is an optimization technique using the tool of memoization to its fullest effect.
- You can see memoization as a subset of DP. Normally, you can use DP to solve problems that are applicable to memoization. However, DP solvable problems are not always suitable for memoization due to possible stack overflow.
Dynamic programming and other similar techniques are powerful problem-solving solutions. If you want to become more competitive in the computer science world, you need to dive deeper to master DP. For further professional programming-related information, Google algo.monster to see what you can find. This website will help you a lot.