Frequently asked questions
What does the self-assessment consist of?
The self-assessment comprises an online questionnaire and the analysis of its results, consisting of both an executive report and a detailed report. The questionnaire is based on a knowledge management framework developed by DPM Research. This framework looks at the IR-related activities and analyzes the usefulness and purposes of its content, the factors influencing IR performance, and the competencies of the office. DPM Research will work closely with your IR office to identify key stakeholders who will be invited to participate in the survey.
What can we expect to receive in the executive report and the detailed report?
The executive report focuses on the competencies required to perform IR activities, including a comparison with peer institutions.
The detailed report provides frequency distributions for the level of development in each competency. This analysis includes competency development across each of the organizational sectors, an examination of the resources that support IR activities, and the perceived usefulness and relevance of the information produced by IR offices.
What is different about this self-assessment from other IR surveys?
Existing surveys tend to focus on an inventory of IR activities, occasionally touching on areas of work and their purposes but without a clear distinction. TIRSA is the most comprehensive and structured assessment available. It clearly delineates sectors, functions, usefulness, relevance, and factors of influence and is the only tool assessing IR competencies.
Who is asked to complete the questionnaire and how do we identify participants?
The IR Director completes the primary questionnaire about internal functioning. A second questionnaire targets stakeholders’ perceptions. Stakeholders are persons who know the IR office, its services, and its products. The choice of which stakeholders are invited to answer is a joint effort between the IR Director and DPM, taking into account the organizational structure of the institution.
Are our responses confidential? Will participants be identified?
Will DPM Research publicly release the results of the self-assessment?
DPM will not publicly release the results. Nevertheless, by participating in this project, you will be contributing your data to a benchmarking dataset. When producing the benchmark results, your data will be anonymously aggregated with a group of other participants.
How can smaller institutions benefit from the self-assessment?
The assessment process is designed to accommodate all institution sizes. Although smaller institutions may cover a more restricted range of IR products, it is still important to know what competencies are used in completing these products and to identify your office’s potential for growth in these processes.
My institution does not have a dedicated IR office per se. Can we still participate?
Institutions that do not have a dedicated IR office still produce reports and analyses out of other units such as the Office of the Provost, the Registrar, Finance or Information Technology offices. Nevertheless, the competencies supporting these reports and analyses remain the same and will be captured in the assessment process. DPM will work with your institution to identify the best participants to complete the primary questionnaire.
How was the TIRSA approach validated?
TIRSA proposes an approach that was developed based on numerous years of experience working in Institutional Research, integrating literature from Canada and the United States. The framework was piloted in Canada. The first full iteration of the model was validated using a Delphi approach (link to paper) and was successfully implemented in a nationwide study in Chile. The results of the Chilean project were used to further refine the instrument. Additional revisions were performed following its implementation within a university system in the United States.
I am unfamiliar with the notions of IR competencies or organisational sectors and functions. Where can I find definitions of these dimensions?
A quick overview is available in the glossary. For a webinar of the framework, its components, and how it structures the IR assessment, click here. For an in-depth description of the framework, you can refer to the publication Díaz, V.E., Mercier, P., Pinsent, C. (2017). A knowledge management framework for institutional research. In J. Huisman & M. Tight (Eds.), Theory and Method in Higher Education Research, Vol 3, pp. 155-179, likely available in published format through your library or here as an Author Accepted Manuscript.
How long will the questionnaire take to complete?
Answering the questionnaire may take between one hour for stakeholders and two hours for IR Directors.
Do I have to complete the questionnaire all at once or are my responses saved?
You may answer a portion of the questionnaire, pause, and return to it at a later time.
I made a mistake on a previous question; can I go back and edit my responses?
Yes, you can return to previous answers and modify them before continuing or completing the questionnaire.
I cannot answer a question; will this affect our results?
If you cannot answer one or more questions, please skip these. The lack of answers may limit certain interpretations and comparisons; however, the inability to answer a question in itself will be analysed to help shed light on its impact, if any, toward the development of information production.
Our office does not perform an activity mentioned; how should I account for this?
When an IR office does not perform an activity, the case will be analysed according to two possibilities:
- If it is performed elsewhere, is this the best choice or could the institution benefit from re-locating or co-locating the activity in IR?
- If the activity is not performed at all, is it because the institution’s mission does not require it or should it be considered desirable for future IR development (including a consideration of resources)?
When can we expect results from the analysis?
Specific timelines will be discussed with each of the participating institutions.