A resource kit for evaluating community IT projects
Evaluate IT logo

Examples using the EvaluateIT resource kit

Here are examples of how community groups can use EvaluateIT to help them review an IT project in their community.

[Note: These are all fictional examples, even where they appear to resemble actual situations.]


example iconExample 1: the Westana review of IT projects

by Jill Korgan, Secretary, Westana Shire Development Association

Background

This is an account of how people in our Westana community worked together to find ways of improving our community IT (information technology) resources and skills. We've been doing this through reviewing and evaluating some of our main IT projects. This story explains how we went about this from when we first thought about doing the review to when we completed it.

The Westana Shire has been badly hit in recent years by drought and bushfires. However, we were lucky in having some enthusiastic people who were determined to make sure the district held together and grew rather than declined. They included Councillors and staff in the Westana Shire Council, our local library staff, school principals and teachers, and members of our many community organisations and sporting clubs.

For the area to flourish, our group believed that people in the different areas of the Shire needed to keep in touch more and work together better to make the most of the resources in our community. We were interested in the possible benefits of using the Internet for communicating and sharing information, and also in training courses on how to use new computer technologies.

So when funding for training in IT became available, we looked into it and encouraged other Westana folk to join us. Among other things, we learned how to set up our own web portal – the Westana Shire website. Community groups also developed their own websites and home pages to let others, within the Shire and beyond, know more about our area and what was happening in our community. They included home pages for a local tourist attraction, a rural women's group and our local history club, and a Community Futures Network website.

Two other examples of community website projects that we found useful were:

Note: Westana is a fictional community but parts of this story have been adapted from actual examples of rural communities reviewing their IT projects.

^ top

step 1 iconStep 1: We plan the review

Who wants to do the review?

Our story begins when Helen Park, one of our Councillors in the Westana Shire Council, thought that the IT projects which had begun with such enthusiasm might come to a halt unless something was done to sustain them. Councillor Park thought that a review of the current projects was needed and wanted to get the community involved.

David Cordell, the Council's Community Development Officer got together a small group of other interested people, including me, to come up with some ideas about how we could generate interest in the review project. We suggested holding information sessions and a workshop to see how much interest there was in the community. We thought it would be more successful in the long run if we could get a mixed group of people from nearly every sector of the community to join us.

After the information session and the workshops, we found we had a group of committed people who were keen to review the IT projects we were involved in. We had mixed backgrounds, experiences and skills. A few were familiar with IT and some of us knew very little, but saw its potential for our families, our businesses and our community. Some were from Westana, the main town in the Shire, and some were from smaller towns in the Shire, or from rural properties.

What IT project(s) are we reviewing?

At first it wasn't easy to decide what we would review. The Westana Shire website had grown tremendously in so many directions that it was now called a community portal. It had many home pages of dozens of different community groups and included online discussion groups for some interest groups.

Several different training courses in IT had been held. They included an Internet training course at the library run by Carol McLean, the Council IT Support Officer, and a computer basics course held at the Westana State School by Alan Thompson, a teacher at the school. The library course was funded by the Council and the School course was held with State government funding obtained by the Westana Shire Development Association.

The situation had become very complicated with dozens of people being affected. We were concerned that some of those affected would not be interested in being involved in the review.

In the end, we decided to concentrate on the IT projects which seemed most manageable, given the people who were interested in the review project. They were:

  • the website of the Westana Community Futures Network, and
  • the IT training courses held at the library and the Westana State School.
Why are we doing the review / evaluation?

There were different reasons why each of these projects needed to be reviewed.
All those involved with the Westana Community Futures Network agreed that, once it had been set up, our website had continued to grow without much further thought about where it was going. Was it still meeting our original goal of inspiring our community to make the most of learning opportunities to grow and prosper? Would it help in our goal of encouraging people to stay in the community rather than move elsewhere?
We also agreed that we should look at the main IT training courses to see how well they were working, on a number of levels. For example, we needed to know:

  • if they were worth the time, money and effort people were putting into them
  • if those who had taken the courses had found them useful, and
  • why many groups in the community didn't attend them, and whether there was anything we could do about this in the future.
What risks could be involved in doing this review?

Before we undertook the IT projects review, we needed to think about any risks that could be involved and who could help us identify and manage them. Some of the risks we considered were:

  • having the project fail because we didn't make sure that we had enough people interested in the project who would do whatever needed to be done, when it needed to be done, and wouldn't be likely to be transferred to another town mid-way through the project
  • having the project fail because we couldn't get the resources we needed, for example, to do a survey that had to be written, printed and mailed out to hundreds of people
  • not getting reliable information or people being upset because we didn't make sure that survey information was collected anonymously, and that confidentiality was maintained in the review report by not naming and quoting anyone without their permission.

At our first meeting David, the Community Development Officer, helped us think through the possible risks and how we could access the resources we needed. We later consulted Dr Sally Green, a researcher from a university in our region, when we were planning the survey and interviews as part of our review.

< back to step 1 | ^ top

step 2 iconStep 2: We involve people in the review

Who else should we involve in the review?

Once we were clear about the what and why of the review, the next step was to have people from all possible sectors of the community involved in the review, especially if they could benefit from the IT projects or from using IT.

From the beginning, we had kept on widening the circle of people we should include.

  • The first meeting had involved only two people from the Westana Shire Council: Councillor Helen Park and David Cordell, the Shire Council's Community Development Officer.
  • They had formed a small steering group by inviting three other people to join them: Alan Thompson, a teacher and IT trainer from the State School, Carol McLean, the IT Support Officer, and myself, Jill Kogan (I'm the Secretary of the Westana Shire Development Association).
  • David had arranged a meeting of leaders from the various townships in our Shire where they discussed the proposal to review IT initiatives in the Shire. They had decided to hold a community workshop and invite all interested people, including anyone who might have been interested in using the Internet and other IT (for purposes such as learning, finding information, promoting a product or service, or keeping in touch with family and friends).
  • Twenty people came to the workshop (twelve women and eight men). We talked about our IT projects and came up with some new ideas about how we could use the Internet. We agreed that doing a review could help us develop our ideas and improve our IT projects.
  • David then presented and discussed ideas at meetings of other community groups and at a meeting of school principals from the district.
  • In the end, David agreed to facilitate a group of twelve volunteers (eight women and four men) who would be the review steering group. They would plan and coordinate the review with help from other people. Our steering group included community leaders, good community networkers, an IT trainer, a retired couple, and an unemployed youth. There were seven people from Westana, three from other townships, and two from properties.
What will encourage them to take part?

We thought of ways of encouraging as many people as possible to take part in the review, such as:

  • telling community leaders about the project and obtaining their help and ideas about getting other people interested
  • holding a community meeting at which people could find out more and ask any questions they had about the review
  • thinking about the best times and places to get the information to people, such as at community meetings that were happening around that time
  • deciding who would be a good person to present the proposal for the review at such meetings (we decided on David, who already knew many people across the Shire, and was enthusiastic about the project).
What can I/ we contribute to the review?

People had volunteered to be on the steering group and others joined us later to help with the survey. There were several ways we could each contribute to the review. For example, different people were good at:

  • networking and keeping people in touch with each other
  • organising meetings and workshops
  • taking notes at meetings and circulating them
  • keeping everyone focussed during the workshops
  • finding useful information on the Internet and at our local library
  • designing surveys.

People gave us feedback about how useful they found the website and the IT training courses. We all gave ongoing feedback on how the review was progressing, any problems that were developing, and what seemed to be working well and what wasn't working so well. We all found ways to tell others about the project and get them involved. They included telling our families, friends and workmates, and getting information published in community newsletters, the local newspaper, and on the Shire website.

What else do we need to do a useful review?

We had to work out what to include in the review. Statistics that we obtained from a recent Shire Council report on their IT training courses were useful in helping us work out what information we already had and what we still needed to collect.

Because we decided that we wanted to do a survey, we needed access to printing, stationary and postage costs. The Council and schools helped us with this, as well as with word-processing the survey itself and printing it.

For a useful review, we also needed to think about which people were likely to be willing and able to help with such things as:

  • collating and folding the surveys to mail them out
  • doing letterbox drops of surveys in town
  • handing surveys out at community meetings and events
  • adding up the numbers for each response on the surveys, and working out the percentages to present to everyone at the following workshop
  • writing reports on our findings.

< back to step 2 | ^ top

step 3 iconStep 3: We do the review

What are the goals of our IT project?

The first step in doing our review was asking what we wanted from our IT projects. Why had the website been set up and what were our goals of the Community Futures Network website? Was the way we planned and designed it effective? What specifically did we think the website and the IT training courses would achieve for ourselves, for other people and groups, and for the community as a whole?

We remembered that one of our goals for setting up the Westana Community Futures Network website had been to become better known in the community. Other people could see what we were aiming to achieve and become interested in contributing to helping create a vibrant community with a future. They could learn more about us from the links to our personal home pages, and contact us easily using the links to our email addresses. We thought this would encourage people to join in our activities, including our online discussions.

The goals for the Internet training course in the local library were to help us become more skilled in using the Internet to efficiently search for information, to send and receive emails and attachments, and to create simple home pages.

The goals for the computer basics course at the State School were to familiarise people with computers so that they would feel more comfortable and confident with using their home computers and the Internet. The aim was for participants to learn basic word processing, and how to create spreadsheets, format documents, manage files, and other basic skills.

The long-term goals of these IT projects included developing skills and tools to help people in their work and leisure, to enable community groups to work together better, and to know more about and join in with community activities. We thought our IT projects could create more opportunities for learning, employment and business development. This could encourage people to stay in, or even move to, the thriving 'learning community' we envisaged.

How are we trying to achieve our goals?

Next we looked at how we had been trying to achieve these goals through our Community Futures Network website and our IT courses. What had we actually done and what were we doing? We noted down everything we had done, including all the major and minor things that went into the projects. There was too much to list here but, in brief:

  • For the Westana Community Futures Network, we used the Shire's resources to set up a website which included an online discussion list. We also used the skills we had learnt in the IT courses to create links on the website to our own personal home pages. We used the website to post notices about activities, to discuss issues and proposals, to answer questions, and to circulate notes from meetings and workshops.
  • For the Internet training course, we organised with Carol McLean, the IT Support Officer, to run four half-day short courses on Saturday afternoons and four weekly courses on Wednesday evenings in the library. They were offered at four different levels, so that different groups of people would feel comfortable attending.
  • The computer basics course at the school was held for three hours over eight weeks after school on Monday afternoons. Alan Thompson, a teacher from the school, organised and ran the course and made the computer room available. Students helped out.
How well are we reaching our goals and how do we know?

But how well had we done? This was the main point of the review - what all the previous work had been building up to. So we wanted to do it well. How could we assess how well our IT projects had achieved their goals? This took several workshops to organise, and included:

  • deciding on the main questions we wanted the review to answer
  • brainstorming ideas about what methods to use in the review
  • holding focus groups to collect feedback
  • planning and carrying out a survey that gathered feedback on both the website and the IT training courses
  • interviewing a range of people such as the IT trainers and the website manager
  • gathering all the data together and analysing it.

David Cordell, the Community Development Officer, helped us run the workshops and focus groups. Members of our group searched the Internet and did some reading to learn more about developing surveys and holding focus groups. Dr Sally Green, a university researcher, gave us advice on the design of our survey and the focus group and interview questions.

(see Suggested ways of using EvaluateIT on this website to get some idea of what's involved in running workshops and using IT to collect feedback and tell people about your review project. The Resources page on this website also has links to information sheets on methods such as brainstorming and focus groups, as well as useful toolkits and publications).

We planned our survey, focus groups and interviews to find out:

  • what people thought worked well and not so well with the IT training courses and the Community Futures Network website and how they could be improved
  • who didn't know what IT courses were available and when and what IT resources were not being used as much as they could have been
  • what problems existed (such as slow connections to the Internet, not enough time to access the website and send emails, courses being on at the wrong times, and a lack of resources to maintain the website)
  • what needs were not being met and whose needs were not being met.

When we put together everything we had learnt from the review, we were able to see what the main problems were, and which people and groups in our community were missing out. We also had a range of suggestions on how to improve both the Community Futures Network website and the IT training courses. Among other things, we found out that:

  • business people had been 'too busy' to take up the opportunities offered to them
  • men were less interested in the IT projects compared with women and were not very well represented in the review
  • many of those who were most disadvantaged (e.g., elderly pensioners, people from non-English speaking backgrounds, indigenous people) were not taking part in IT training and did not know that they could get free access to IT training and the Internet in the library.
How could we do our project better?

We had gathered many good ideas about things we could do to improve our IT projects, such as:

  • using a wider range of ways of promoting the IT training courses and the free Internet access in the library
  • running the courses at different times to suit different groups of people
  • holding IT courses in retirement villages and places that disadvantaged groups find more comfortable, using mobile training equipment from the local Open Learning Centre
  • making better use of the resources we had (e.g., computers at local schools that were not being used after school hours)
  • promoting our website better using the local radio station, local newspapers and the Council newsletter
  • including more information about our activities on our website, using a more interesting and user-friendly layout and wording.

< back to step 3 | ^ top

step 4 iconStep 4: We review results and make the changes

Have we reviewed our evaluation results and method?

We had gathered plenty of interesting findings, but, before we acted, we needed to check them out:

  • to see if they made sense
  • to see if they had come from a good mixture of people or not (for example, women and men, younger and older people, people with and without IT skills and experience, disadvantaged and more advantaged groups)
  • to decide how well the findings related to our goals.

We also wanted to think about what we had learned about doing reviews so that we could do the next review better. We held a workshop to look at:

  • how we managed any problems that cropped up, such as our IT Support Officer leaving town in the middle of the review
  • how the various methods we used could have been improved, for instance by doing some telephone surveys of people who did not return the printed surveys because of low literacy levels
  • which other people we could ask to help us with our review next time.
How will we let others know the results of our review?

We told as many people as we could about the review and what our main findings were, to try to get more of the community involved in the IT projects and thinking about how IT can benefit them and our community. We used several methods:

  • a feature article in the Westana Shire newspaper
  • a flier in the Westana library
  • a note in school newsletters
  • talks at several meetings of community groups
  • an interview in a talk-back program on the local radio station
  • publishing a brief report on the Westana Shire website.
What do people think about the review results?

This publicity made more people aware of the review results and we received more feedback about the review. This included feedback from those involved in the Community Futures Network or who had done the IT training courses (or missed out), and many others in the community, both in town and on properties. We also started to get some interest from potential sponsors such as an Internet service provider.

Most people commented that our review results were interesting and useful. They said it had increased their awareness of the opportunities that IT could provide and the ways that local IT resources could be better used. Councillor Park said she was pleased with the quality of the report and the very clear and effective way that the findings had been presented.

How will we make the changes that are needed?

The next stage was to make the changes that we decided needed to be made, based on all the ideas and suggestions. This took several steps:

  • deciding which changes should have priority and which were less important
  • deciding who could and should make the changes
  • finding the funding we needed to run more courses and provide more publicity for them
  • putting the information we needed into a short report to present to funding organisations to support our applications
  • deciding on a timeline for the changes and on who would do what.

As we are carrying out these changes we are continuing to have meetings and to think about going through this review process again in the future. We are thinking of making more use of email and our website to gather feedback and holding teleconferences to link some remote people who cannot travel to our meetings. We know that next time, it will be easier and quicker because we will use the step-by-step process and questions on the EvaluateIT website to keep us focussed on what we are doing and why.

We feel more confident in regularly reviewing our IT projects and we have learned a lot about making better use of our IT resources to help the community develop in the ways we want. We have all learnt new skills, not just in IT, but also in how to facilitate meetings and workshops, conduct surveys and analyse them, write reports, and speak in public about projects in the Westana community.

The review process has created strong new connections between those involved. Some of the barriers between different groups are also starting to break down as more people talk to each other and find out what the different groups in our community are doing and how we can work together better.

Note: Westana is a fictional community but parts of this story have been adapted from actual examples of rural communities reviewing their IT projects.

< back to step 4 | ^ top


Example 2: Review of IT training

A regional community group reviews the impact of IT courses offered in their community by the BHP-Billiton Skillsnet Roadshow Queensland.

step 1 iconStep 1: Plan the review

A small group of librarians, IT trainers, some past course participants, two council staff (an IT service desk consultant and a youth worker) get together to plan the review. It is now two years since the BHP-Billiton Skillsnet Roadshow Queensland visited their city. They want to find out how useful the courses proved to be and what follow-up (if any) is needed. Before they go ahead they discuss any risks that might be involved and how they plan to minimize them.

step 2 iconStep 2: Involve people in the review

They decide they need to widen the group to involve staff representatives from several groups. These include the Town Council, and local IT businesses/industry partners who provided support for the courses. They work out ways to get these folk interested and involved. They also consider what resources, talents and contacts they themselves have to contribute, and what else the group needs to carry out an effective review. 

step 3 iconStep 3: Do the review

First they make sure they agree on the goal for the review. They decide they all want to see 1) whether the courses met the needs of the local community, and 2) whether more courses are needed and if so, which ones. They discuss what the needs of the local community are and how they can investigate whether the Roadshow courses met these needs or not. They decide to look at statistics about how many people attended courses and which courses they attended. They also decide to carry out a survey to find out how useful the courses were for local people, and whether the associated course materials on the Roadshow website were being used. Then they work out how they will do this and who will do what. Finally, when they have their findings, they discuss what needs to be done to address the community's unmet training needs. They find that the earlier courses have created a huge demand for further IT training and that other groups in the community are both willing and able to help the local library meet this demand.

step 4 iconStep 4: Review the results and make the changes

After checking that their survey methods are thorough and fair, they discuss how to let the rest of the community know, how to get feedback about what others think, and how to make the changes that need to be made.

^ top


Example 3: Review of a Learning Network Queensland centre

A Learning Network Queensland (LNQ) Centre in a regional town in Queensland used EvaluateIT to do a review of how the services available matched up with the needs of users and would-be users. They went through the four steps (1. Plan the review, 2. Involve people, 3. Do the review, and 4. Review results and make the changes) and found that they could make much better use of the resources available by planning to accommodate the lifestyles of those who want to use the Centre. For example, they created a new booking system and set up a roster of local people who could be contacted when problems arose.

^ top


Example 4: Review of an email discussion list

This email discussion list looked like a promising idea when a few Indigenous students got together to set it up. The idea was to have a support network for Indigenous students living away from their communities while they attended schools, TAFEs and universities in the city. However, it didn't really get off the ground. They decided to use the EvaluateIT resource kit to plan and carry out an evaluation of the email discussion list. They used the email list itself to share ideas as they went through the four steps (1. Plan the review, 2. Involve people, 3. Do the review, and 4. Review results and make the changes).

In the process, they found that there was, in fact, a real need for such a list. They shared lots of great ideas about how to contact new students and have them use the support available on the list to sort out some of the teething problems they had as they settled into their new lives in the city and at their various educational institutions. It also became a place where notices for get-togethers and other events could be posted.

^ top