Course detail

Stochastic Processes

FEKT-MPC-NPRAcad. year: 2024/2025

The course includes an introduction to the theory of stochastic processes types. Therefore, it starts with repetition of necessary mathematical tools (matrices, determinants, solving equations, decomposition into partial fractions, probability). Then we construct the theory of stochastic processes, where we discuss Markovský processes and chains, both discrete and continuous. We include a basic classification of state and students learn to determine them. Great attention is paid to their asymptotic properties. The next section introduces the award transitions between states and students learn the decision-making processes and their possible solutions. In conclusion, we mention the hidden Markov processes and possible solutions.

Language of instruction

Czech

Number of ECTS credits

5

Mode of study

Not applicable.

Guarantor

doc. RNDr. Jaromír Baštinec, CSc.

Department

Department of Mathematics (UMAT)

Entry knowledge

We require knowledge at the level of bachelor's degree, i.e. students must have proficiency in working with sets (intersection, union, complement), be able to work with matrices, handle the calculation of solving systems of linear algebraic equations using the elimination method and calculation of the matrix inverse, know the series and their sums, know the graphs of elementary functions and methods of construction, differentiate and integrate of basic functions.

Rules for evaluation and completion of the course

Requirements for successful completion of the course are provided in annual public notice. Students can obtain:
Up to 30 points for computer exercises that can be obtained a written test (20 points) and 10 points for activity assessment exercises.
Up to 70 points for the written final exam. The test contains both theoretical and numerical tasks that are used to verify the orientation in the problems of stochastic processes and their applications.
.
Computer exercises are compulsory. Properly excused absence can be replaced by individual homework, which focuses on the issues discussed during the missed exercise.
Specifications of the controlled activities and ways of implementation are provided in annual public notice.
Date of the written test is announced in agreement with the students at least one week in advance. The new term for properly excused students is usually during the credit week.

Aims

The aim of the course is to provide students with a comprehensive overview of the basic concepts and results relating to the theory of stochastic processes and especially Markov chains and processes. We show possibilities of application of the decision-making processes of various types.
The student is able to:
• Describe the basic properties of random processes.
• Explain the basic Markov property.
• Build an matrix of a Markov chain.
• Explain the procedure to calculate the square matrix.
• Perform the classification of states of Markov chains in discrete and continuous case.
• Analyze a Markov chain using the Z-transform in the discrete case and the Laplace transformation in the continuous case.
• Explain the procedure for solving decision problems.
• Describe the procedure for solving the decision-making role with alternatives.
• Discuss the differences between the Markov chain and hidden Markov chain.

Study aids

Not applicable.

Prerequisites and corequisites

Not applicable.

Basic literature

BAŠTINEC, J.; SVOBODA, Z. Náhodné procesy. Brno: 2011. s. 1-182. (CS)
Prášková Z., Lachout P., Základy náhodných procesů I., MatfyzPress (CS)

Recommended reading

Not applicable.

Classification of course in study plans

  • Programme MPC-BTB Master's 1 year of study, winter semester, compulsory-optional

Type of course unit

 

Lecture

26 hod., optionally

Teacher / Lecturer

doc. RNDr. Jaromír Baštinec, CSc.

Syllabus

1. Random variables, basic notions.
2. Stochastic processes, characteristics of stochastic processes.
3. Discrete-time Markov chains, Chapman-Kolmogorov equations.
4. Homogeneous Markov chains.
5. Regular Markov chains.
6. Absorption chains.
7. Z-transformation, analysis of Markov chains.
8. Continuous-time Markov chains.
9. Poisson process.
10. Chapman-Kolmogorov differential equations.
11. Markov decision processes.
12. Asymptotical properties of Markov chains.
13. Decision process with alternatives.

Computer-assisted exercise

26 hod., compulsory

Teacher / Lecturer

doc. RNDr. Jaromír Baštinec, CSc.

Syllabus

1. Introduction to statistic software.
2. Analysis of random variables.
3. Calculation of characteristics of random variables.
4. Discrete-time Markov chains-applications.
5. Applications and solving of Chapman-Kolmogorov equations.
6. Homogeneous and regular Markov chains-applications.
7. Applications of absorption chains.
8. Analysis of Markov chains by using Z-transformation.
9. Characteristics of continuous-time Markov chains.
10. Applications of the Poisson process.
11. Applications and solving of Chapman-Kolmogorov differential equations.
12. Analysis of Markov decision processes.
13. Asymptotic analysis of Markov chains.