How well do you know your Markov Decision Process and Markov Chains? Because they are topics you will come across on the clinical informatics subspecialty exam.
To better prepare you, we were thinking of putting together a big blog post discussing the topic but quickly realized that there are a lot of resources out there that can probably explain it better than we can (although we are planning on covering the topic more in-depth and with a clinical informatics twist in the future).
We’re also in the midst of writing our full-length practice exam for the boards. It should be released by August and guess what – it will include many questions on Markov topics. Yipee!
p.s if we had to pick favorites or you’re short on time, first take a quick look at the “origin of markov chains” video, then watch PatrickJMT’s youtube series on the topic, and finally play around with them interactively using the “visual explanation of markov chains” page.
List of Markov Resources
In order of difficulty (beginner – advanced)
A Visual Explanation of Markov Chains by Victor Powell
This should be your first stop. On this page, you will get to visually understand markov chains as well as interact with them by changing variables.
http://setosa.io/blog/2014/07/26/markov-chains/
Origin of Markov Chains (Part of Khan Academy’s Modern Information Theory’s Course)
If you’re a visual learner and need some refreshing on Markov Chains, than this video should do the trick.
Markov Chains Parts 1-7
Wow! We love this YouTube Series by PatrickJMT. You’ll work through progressively more difficult problems and brush up on your multiplcation matrices. Each video is around ten minutes long, so it makes for excellent education on-the-go.
https://www.youtube.com/watch?v=uvYTGEZQTEs
Introduction to Markov Processes (a.k.a. Markov Chains)
A simple, yet practical explanation of the topic.
http://austingwalters.com/introduction-to-markov-processes/
Stanford’s Probabilistic Graphical Models Class on Coursera
Check out weeks 2 & 5. This is also a great resource for many other topics in decision science and for other general knowledge that will be handy to know when machines take over our daily lives.
https://class.coursera.org/pgm-003/lecture
Circuit’s of Imagination – Markov Chains
This one gets a little more advanced by throwing some code into the mix. Also, the example problem they give is intriguing.
http://blog.circuitsofimagination.com/2015/02/15/Markov-Chains.html
Markov Chain Monte Carlo Sampling
This one’s pretty advanced and goes into algorithms and lots of python, so if that is your kind of thing, enjoy!
We hope these links provide you with a good starting point for your review. Feel free to add other resources you think would help others in the comments section below.