Split-Half Check of Reliability

This is a kind of test reliability in which one test is split into two shorter tests. Test questions are randomly assigned to each test. After the testing, the two can be compared, and retesting using the other half of the test can be done.

 

ASTD “Measuring and Evaluating” p8

Reliability

Reliability refers to the ability of the same measurement to produce consistent results over time. Some data is inherently reliable (such as a person’s age) and other more subjective data is not very reliable.

Another way of putting it is that reliable tests are free from significant test errors.

Reliability can be interpreted as an index of test results error. Generally, tests with reliability coefficients at or above 75 percent (p=0.75) are considered reliable.

If we were to throw darts at a board, reliability is that the darts are grouped together closely, and not randomly placed on across the board. Validity is that the darts are at the center of the target. Furthering this example:

1. If the darts are clumped together but not in the center, that is reliable, but not valid.
2. If the darts are clumped together (reliable) and in the center, that is both reliable and valid.

 

 

Source: ASTD’s Measuring and Evaluating, pp. 6-7

Training Evaluation Validity and Reliability

Validity = measuring what you intend to measure.

Types of Validity:

1. Content Validity: the extent to which the instrument represents the training program’s content.
   1. Low Content Validity: the instrument doesn’t represent a true summation of the program content
   2. High Content Validity: the instrument represents a good balance of all the program content
2. Construct Validity: the degree to which an instrument represents the construct it is supposed to measure
   1. The abstract variable, such as skill or ability, is the construct
   2. Can be defended through expert opinion, correlations, logical deductions, and criterion group studies
3. Concurrent Validity: the extent to which an instrument agrees with the results of other instruments
4. Criterion Validity: the extent to which the assessment can predict or agree with external constructs.
5. Predictive Validity: the extent to which an instrument can predict future behaviors or results.

Source: ASTD Learning System: “Measuring and Evaluating” p. 6

Training Measurement Process

All evaluation models contain the following steps:

1. Identifying evaluation GOALS
2. Developing an evaluation DESIGN AND STRATEGY
3. Selecting and constructing MEASUREMENT TOOLS
4. ANALYZING DATA
5. REPORTING DATA

 

Source: ASTD Learning System: “Measuring and Evaluating” p. 5

Benefits of Training Evaluation

These are some benefits for the instructor.

1. It shows professionalism of the trainer.
2. It proves the results of the training.
3. It focuses the training.
4. It validates performance gaps and learner needs.
5. It validates if training is the solution.
6. It provides a cost-benefit analysis for training.
7. It gains access to resources for the trainer.

 

Source: ASTD Learning System: “Measuring and Evaluating” p. 4-5

Performance Agreement Principle

Every learning objective should have a corresponding evaluation task.

Evaluation tasks can be FORMAL or INFORMAL

Objectives should match evaluation tasks closely in two important areas: BEHAVIOR and CONDITION

Behavior describes the anticipated outcome of the training, while condition describes the circumstances under which performance is to take place.

When these elements match, an objective is said to have a PERFORMANCE AGREEMENT.

 

The performance agreement principle ensures consistency between what’s expected of learners in the objective and what they’re required to do in an evaluation.

 

Source: ASTD Learning System: “Measuring and Evaluating” p. 4

Benefits of Training Evaluation

1. Determining business impact, the cost-benefit ration, and the ROI for the program.
2. Improving the design of the learning experience
3. Determining whether the objectives of the learning experience were met and to what extent
4. Determining the content’s adequacy (more job related, more challenging, support objectives)
5. Assessing the effectiveness and appropriateness of instructional strategies
6. Reinforcing learning
7. Providing feedback to the facilitator
8. Determining the appropriate pace and sequence
9. Providing feedback to participants about their learning
10. Identifying which participants are experiencing success in the learning program
11. Identifying the learning being used on the job
12. Assessing the on-the-job environment to support learning.

Source: ASTD Learning System: “Measuring and Evaluating”

Instant Video Presenter is an easy way to create engaging learning

I found a product on the Internet that looked really innovative. It is called Video Instant Presenter and it is available from the people at EJ4.

The idea is simple: You can record video of the trainer standing in front of a green screen, and then on your PC you have the PowerPoint or other media. The program will magically make what is on your computer screen show up as a background to the person doing the training. This is an inexpensive program and could be the answer for creating content that merges the image of a real live instructor with your PowerPoint, for asynchronous learning. I am planning on ordering the program and will post how it works.

Low Cost: The cost for this product is incredibly cheap: If you buy it with a camera included it is less than $250. If you already have a camera, you can get it without the camera for under $150. This price includes the Green Screen. You can download a trial version for free, but the free trial doesn’t include the green screen.

The link for more information is http://affiliate.instantvideopresenter.com/aw.aspx?A=78

7 Rules for Great PPT Presentations

May 7, 2010

Having an understanding of what makes for an awful PowerPoint presentation can make it intimidating to create your own. Who wants to be the creator of one of those presentations?

In our consulting work at McCulley/Cuppan, we’ve sat through, created, and helped others create many presentations (including ones to FDA and other regulatory agencies), so we have some experience with good and bad PPT.  Here’s a list of rules we’ve developed over the years for designing effective presentations:

1. Adopt your audience’s mind–it’s always about them, not you
   

   When you are the subject matter expert and have done lots of research, you may feel compelled to share it all in the PPT slides. Avoid this. Instead, focus on what your audience is most interested in. (Your knowledge will come through in the way you discuss the topic and answer questions.)
   

2. Clarify your intentions
   

   How will you proceed with the presentation? Will there be a Q&A session? If so, when?  Let your audience know upfront how much time you will take and the basic outline of how the time will be spent.
   

3. Simplify, simplify, simplify
   

   Simpler is always better.  Simple design, simple text. As Hans Hofmann said nearly 100 years ago: “To simplify, eliminate the unnecessary so the necessary can speak.”
   

4. Embrace limitations and practice restraint–know when to stop
   

   Once again, all of your knowledge does not need to be demonstrated in this one presentation. Respect the time and concentration limits of your audience.
   

5. Do like the pros do–script, storyboard, create
   

   Planning is the key to effective presentations. Work with the end in mind even before you build the first slide and the content of your presentation will be more compelling.
   

6. Think communication not decoration
   

   PowerPoint is a tool to help you convey knowledge; keep the slide design simple so the focus stays on the content.
   

7. All presentations are storytelling, so become a master storyteller
   

   Learn from the master storytellers around you, people whose presentations you’ve enjoyed attending. If you don’t have any nearby, learn from well-respected presenters such as Garr Reynolds, Steve Jobs and Guy Kawasaki. Tell a tale that engages the audience and makes them want to listen and learn from you.
   

 

Source: Knowledge Management and Communication in the Life Sciences

Notes from “Rules of Thumb” by Alan M Webber

 

Book Title: Rules of Thumb: 52 Truths for Winning at Business Without Losing Your Self

I got it from Amazon.com

The 52 RULES OF THUMB

01. When the going gets tough, the tough relax.
02. Every company is running for office. To win, give the voters what they want.
03. Ask the last question first.
04. Don’t implement solutions. Prevent problems.
05. Change is a math formula.
06. If you want to see with fresh eyes, reframe the picture.
07. The system is the solution.
08. New realities demand new categories.
09. Nothing happens until money changes hands.
10. A good question bats a good answer.
11. We’ve moved from an either/or past to a both/and future.
12. The difference between a crisis and an opportunity is when you learn about it.
13. Learn to take no as a question.
14. You don’t know if you don’t go.
15. Every start-up needs four things: change, connections, conversation and community.
16. Facts are facts; stories are how we learn.
17. Entrepreneurs choose serendipity over efficiency.
18. Knowing it ain’t the same as doing it.
19. Memo to ladders: Focus on the signal-to-noise ratio.
20. Speed = Strategy.
21. Great leaders answer Tom Peters’ great question: “How can I capture the world’s imagination?”
22. Learn to see the world through the eyes of your customer.
23. Keep two lists: What gets you up in the morning? What keeps you up at night?
24. If you want to change the game, change the economics of how the game is played.
25. If you want to change the game, change customer expectations.
26. The soft stuff is the hard stuff.
27. If you want to be like Google, learn Megan Smith’s three values.
28. Good design is table stakes. Great design wins.
29. Words matter.
30. The likeliest sources of great ideas are in the most unlikely places.
31. Everything communicates.
32. Content isn’t king. Context is king.
33. Everything is a performance.
34. Simplicity is the new currency.
35. The red Auerbach management principle: Loyalty is a two-way street.
36. Message to entrepreneurs: Managing your emotional flow is more critical than managing cash flow.
37. All money is not created equal.
38. If you want to think big, start small.
39. “Serious fun” isn’t an oxymoron; it’s how you win.
40. Technology is about changing how we work.
41. If you want to be a great leader, first get real about leadership.
42. The survival of the fittest is the business case for diversity.
43. Don’t confuse credentials with talent.
44. When it comes to business, it helps if you actually know something about something.
45. Failure isn’t failing. Failure is failing to try.
46. Tough leaders wear their hearts on their sleeves.
47. Everyone’s at the center of their map of the world.
48. If you want to make change, start with an iconic project.
49. If you want to grow as a leader, you have to disarm your border guards.
50. On the way up, pay attention to your strengths; they’ll be your weakness on the way down.
51. Take your work seriously. Yourself, not so much.
52. Stay alert! There are teachers everywhere.