Voice and Speech Quality Perception: Assessment and Evaluation

With respect to the group of listeners/test persons/customers the following aspects were discussed:
What is the listeners' general attitude to technology and especially to speech technology?
How much knowledge do listeners have of speech technology?
What experience of technology do they have?
How do listeners assess technology's potential beforehand?
How do they assess improvements in performance?
How do they assess a system's degree of innovation?
What do they consider to be the most useful features of new technology?
With respect to the group of researchers/system developers/suppliers:
What are the aims of researchers, system developers and/or suppliers?
Which difficulties immanent to the task do they have to face?
Are there tasks to which there are principally no solutions which have an effect on the intended product quality?
Do the researchers/developers limit their research/development/optimization to certain sub-functions of the system?
Are partial short term solutions necessary? If so, which ones?
How do such unavoidable short term solutions manifest themselves in the system's behavior?
What loss of quality has to be accepted because of this?
Are any of the expected concessions to quality adequately described?
Is the system's functionality sufficiently documented?
With respect to the system classification:
Which system is involved?
speaking machine
speech machine
translated text-to-speech synthesis
dialog-to-speech synthesis
concept-to-speech synthesis
text-to-speech synthesis
phoneme-to-speech synthesis
manually driven synthesis
With respect to the system in- and output:
What features are inherent to the input (e.g. complexity of the text)?
Which speech style does the input require?
Which features does the system...