Course detail

Audio and Speech Processing by Humans and Machines

FIT-ASDAcad. year: 2023/2024

3 day intensive course
Interaction between humans and machines could be greatly enhanced through communication using human sensory signals such as speech. Knowledge of human information processing is critical in the design of such human-machine interfaces.

State doctoral exam - topics:

  1. Which property of human hearing is used in almost all existing techniques for speech recognition.
  2. Describe structure of human ear.
  3. How is frequency analysis of sound done in the ear?
  4. How is the information from ear communicated to human brain?
  5. What is the general tendency of frequency resolution of human hearing? How does it differ from frequency resolution of the Fourier analysis?
  6. What is auditory masking? What it can be good for and why?
  7. What is simultaneous and forward masking in human hearing?
  8. What does loudness of sound depend on?
  9. At which frequencies we hear the best?
  10. Describe some speech analysis techniques that use more advanced knowledge of human hearing.

Language of instruction

Czech

Mode of study

Not applicable.

Guarantor

prof. Ing. Hynek Heřmanský

Department

Department of Computer Graphics and Multimedia (UPGM)

Entry knowledge

Not applicable.

Rules for evaluation and completion of the course

Oral exam.

Aims

The course covers concept of signal as a carrier of information, basic principles of processing of cognitive signals, and introduces selected phenomena in auditory and visual perception.
Students learn how to interpret empirical data, how to incorporate these data in models, and how to apply these models to engineering problems. Emphasis is on active research in auditory modelling that exploits special properties of speech.

Study aids

Not applicable.

Prerequisites and corequisites

Not applicable.

Basic literature

Not applicable.

Recommended reading

Ben Gold, Nelson Morgan, Dan Ellis: Speech and Audio Signal Processing: Processing and Perception of Speech and Music, Wiley-Interscience; 2nd Edition, 2011.
Brian Moore: An Introduction to the Psychology of Hearing, 6th Edition, BRILL 2013.
Simon Haykin: Neural Networks And Learning Machines, Pearson Education; Third edition, 2016.

Classification of course in study plans

  • Programme DIT Doctoral 0 year of study, winter semester, compulsory-optional
  • Programme DIT Doctoral 0 year of study, winter semester, compulsory-optional
  • Programme DIT-EN Doctoral 0 year of study, winter semester, compulsory-optional
  • Programme DIT-EN Doctoral 0 year of study, winter semester, compulsory-optional

  • Programme CSE-PHD-4 Doctoral

    branch DVI4 , 0 year of study, winter semester, elective

  • Programme CSE-PHD-4 Doctoral

    branch DVI4 , 0 year of study, winter semester, elective

  • Programme CSE-PHD-4 Doctoral

    branch DVI4 , 0 year of study, winter semester, elective

  • Programme CSE-PHD-4 Doctoral

    branch DVI4 , 0 year of study, winter semester, elective

Type of course unit

 

Lecture

39 hod., optionally

Teacher / Lecturer

prof. Ing. Hynek Heřmanský

Syllabus

  • Day 1
    Introduction to processing of information-bearing sensory signals such as speech. Fundamentals of information theory and of pattern classification. Fundamentals of speech production. Conventional techniques for speech analysis (concept of short-term analysis, band-pass filtering, fourier-like transforms, cepstrum, linear prediction).
  • Day 2
    Fundamentals of human auditory perception. Perception of pitch and loudness. Spectral and temporal resolution of hearing. Masking in frequency and in time. Some important speech perception phenomena.
  • Day 3
    Introduction to auditory-like speech analysis techniques. Linear discriminant analysis and its use for deriving optimized spectral basis Temporal domain for speech analysis. Dynamic features of speech and RASTA technique. Multi-stream speech recognition. Recognition from temporal patterns and nonlinear discriminant mapping approaches speech.

Guided consultation in combined form of studies

26 hod., optionally

Teacher / Lecturer

prof. Ing. Hynek Heřmanský