#CMEchat 24: How much do we know about linking learning objectives to assessment models?


We had one of our most actionable and practical #CMEchat's this morning - I have already heard from a few folks that are going to make some changes to their planning process, grant writing narratives, and publish some new ways of thinking about this stuff - all good!

Our topics were as follows:

  1. T1: How do you choose the ‘action word’ in your learning objectives? Do you engage in Bloom’s Taxonomy? here are 2 resources: http://t.co/A3VUAkb & http://t.co/jfybBgg   #CMEchat

  2. T2: Bloom suggests levels of levels of learning outcomes from remembering to creating - How do you relate Bloom’s work to Don Moore’s pyramid? #CMEchat

  3. T3: SMART criteria may also be used when crafting learning objectives. What info is needed to support the crafting of SMART objectives. #CMEchat http://en.wikipedia.org/wiki/SMART_criteria 

  4. T4: So now you are locked into your “SMART Bloom” objectives…how/when will you assess whether the objective was met? #CMEchat

  5. T5: What resources do you use to help you link assessment to educational planning and learning objectives? Is this evidence-based? #CMEchat

  6. T6: In a research model SMART objectives can be considered your hypotheses, your assessment plans are your methods. Discuss. #CMEchat

The archive can be found below or downloaded HERE:

From User    —-    Tweet
  • dpguinee          Great #cmechat today...thx to all...was relegated to lurker status
  • cmeadvocate    .@improvecme - ur absolutely correct. in theory the planning table is a great summary. but rarely ever supported by data = #failure #CMEchat
  • DownstateCME @cmeadvocate safe travel, great chat! #cmechat
  • dean_jenkins     And publish then we hope! RT @theCMEguy: Thanks for the convo! Im off to assess outcomes... #cmechat
  • cmeadvocate    Thanks to all for participating in #CMEchat...hope to get to archive while I travel to #Med2...
  • rmtyner @cmeadvocate Thanks for the great #CMEchat safe travels to #med2
  • dean_jenkins     @cmeadvocate ... explaining @BMJLearning is a little longer than a tweet but would be happy to! #cmechat
  • kenny_cox        #CMechat - good chat today - Plenty to digest - Thanks for moderating @cmeadvocate
  • improvecme      #cmechat—Thanks for your insights today—much appreciated.
  • cmeadvocate    @dean_jenkins - can you tell us what @BMJLearning is all about? #CMEchat
  • theCMEguy       Thanks for the convo! I'm off to assess outcomes... #CMEchat
  • cmeadvocate    T6 - i have added the "lack of a research agenda" to the long list of why this is not really a 'profession' #CMEchat
  • theCMEguy       You said it... RT @dean_jenkins: @theCMEguy yes but no but #cmechat
  • improvecme      T6 #cmechat Need universal use of effect size in published outcomes measurement articles to facilitate later metaanalyses.
  • theCMEguy       #TOTD RT @betsymeded: T4 - how about an objective of 'Improve 20% of hypertensive pt's bp by 10 mm Hg on diastolic...' #CMEChat
  • dean_jenkins     @theCMEguy yes but no but #cmechat
  • kenny_cox        True - @improvecme: T6 #cmechat other weaknesses—limited details on instructional design, cursory discussion of limitations.
  • dean_jenkins     TOTD @cmeadvocate on Roth saying that most CME publications overstate their results and are long-winded. #cmechat
  • theCMEguy       Yes, but... RT @dean_jenkins: T6 - a 'research agenda' is something we should be advocate for at #cmechat
  • improvecme      T6 #cmechat other weaknesses—limited details on instructional design, cursory discussion of limitations. Need universal use of effect size.
  • cmeadvocate    so many difference competencies therein...RT @dean_jenkins: T6 - a 'research agenda' is something we should be advocate for at #cmechat
  • dean_jenkins     T6 - a 'research agenda' is something we should be advocate for at #cmechat
  • CMEChat          The real impact of the #CMEchat lies in your action - engage others to participate & please take action on things your have learned! G’day!
  • CMEChat          As we get wrap up, please vote for the tweet of the day (#TOTD) seen during this week's #CMEchat.
  • CMEChat          Thx for participating in #CMEchat. All discussions herein are assumed 2 b personal opinion & not that of current, past, or future employers.
  • dean_jenkins     here here RT @cmeadvocate: t6 - Roth [said] two major flaws in publishing CME outcomes is overstating results and being too long #cmechat
  • cmeadvocate    t6. Roth: there's no relevance in the outcomes of ur programs - 'Who Cares?' - instead, answer WHY learning too place = good pub. #CMEchat
  • dean_jenkins     T6 - interesting perspective. From an educational research / evaluation view that's a very useful way of thinking about it. #cmechat
  • theCMEguy       @cmeadvocate Yes, agree with the "overstating results" point. See it too often #CMEchat
  • theCMEguy       #CMEchat RT @improvecme: @cmeadvocate Table is a reasonable start, but needs practical underpinnings and details given the realities of CME.
  • cmeadvocate    t6 - Roth made the point that the two major flaws in publishing CME outcomes is overstating results and being too long....#CMEchat
  • cmeadvocate    depends....RT @theCMEguy: @CMEChat T6: Hmmm...wouldn't your activity be your methods? #CMEchat
  • CMEChat          t6 - kudos to #CBIgrants for running a publication planning session for CME folks...anyone participate...Roth, Roy, and Patel. #CMEchat
  • theCMEguy       @CMEChat T6: Hmmm...wouldn't your activity be your methods? #CMEchat
  • CMEChat          T6: In a research model 'SMART objectives' may be considered your hypotheses & your assessment plans are your methods. Discuss. #CMEchat
  • kenny_cox        On my iPad @theCMEguy: hanging by my desk #geek RT @cmeadvocate: In Moore's last article w/ the update pyramid levels he... #CMEchat
  • improvecme      @cmeadvocate Have found success with these with several providers, modifications made based on further needs validations.#CMEchat.
  • theCMEguy       I also have that hanging by my desk #geek RT @cmeadvocate: In Moore's last article w/ the update pyramid levels he... #CMEchat
  • cmeadvocate    BTW - any1 know the history of Moore's Levels...go back and read the original book chapter...#CMEchat
  • cmeadvocate    In Moore's last article w/ the update pyramid levels he (and Joe and Harry) included a table connecting possible methods. #CMEchat
  • betsymeded      Depends, but start short term 1st. Like the use of SMART RT @cmeadvocate: when? permanently? RT 'Improve 20% of HTN pt's bp...' #CMEChat
  • cmeadvocate    .@improvecme - IMO, those tables are crutches and rarely applied appropriately... usually fall apart in top left cell...#CMEchat
  • betsymeded      Nope! :-D RT @thecmeguy: @betsymeded Have you ever seen a CME activity you would be comfortable having that objective? #CMEchat
  • cmeadvocate    T5 - any1 have an answer to t5? what resources direct assessment-to-objectives linkages? #CMEchat #beullarbeullar
  • improvecme      T5 #CMEchat Grants docs often contain table that links & spells out these linkages—needs to objectives to desires results to assessments.
  • rmtyner love it RT @betsymeded: T4 - how about an objective of 'Improve 20% of hypertensive pt's bp by 10 mm Hg on diastolic...' #CMEChat
  • cmeadvocate    when? permanently? RT @betsymeded: T4 - how about 'Improve 20% of hypertensive pt's bp by 10 mm Hg on diastolic...' #CMEChat
  • theCMEguy       @betsymeded Have you ever seen a CME activity you would be comfortable having that objective? #CMEchat
  • cmeadvocate    time and date pls, I want 1st one! RT @improvecme: in the works—Practical Assessment of CME Based on Moore's Levels of Outcomes. #CMEchat
  • kenny_cox        Now that is being specific ;) RT @rmtyner: #sarcasm #CMEchat RT @cmeadvocate: I have also never seen an object… (cont) t.co/7Zennnx
  • betsymeded      T4 - how about an objective of 'Improve 20% of hypertensive pt's bp by 10 mm Hg on diastolic...' #CMEChat
  • CMEChat          T5: What resources do you use to help you link assessment to educational planning & learning objectives? Is this evidence-based? #CMEchat
  • NixTheNext       RT @cmeadvocate: also never seen an objective stating: "immediately after Q&A learners will answer identical Q's 30% + correctly." #CMEchat
  • DownstateCME @dean_jenkins , Bloom's taxonomy is cognitive. Moore's outcomes framework is process-based....few correlations but workeable #cmechat
  • rmtyner #sarcasm #CMEchat RT @cmeadvocate: I have also never seen an objective stating: "immediately after the Q&A lea… (cont) t.co/UZJOvK1
  • cmeadvocate    .@improvecme - this timing issue is critical, IMO, when should you assess change...if that change is real and permanent? #cmechat
  • cmeadvocate    I have also never seen an objective stating: "immediately after the Q&A learners will be answer identical Q's 30% more correctly." #CMEchat
  • improvecme      T4 #cmechat Timing for measuring if objective met could fall into 3 general categories: during CME, at end of CME, and some time after CME.
  • DownstateCME @theCMEguy @betsymeded, such generic word should be ban fm all cme apps #cmechat
  • cmeadvocate    T4 - Who will be the 1st to publish a model for assessment based on a) Bloom's work & 2) Moore's levels? #CMEchat (reread Moore's latest)
  • theCMEguy       T4: Depends on your objective. Knowledge/learning could be assessed immed after program. Performance should be assessed "later" #cmechat
  • rmtyner #TOTD #CMEchat MT @improvecme: you've never seen "Exhibit memory of previously-learned materials by recallin… (cont) t.co/iitthob
  • cmeadvocate    ;-) RT @improvecme: you've never seen "Exhibit memory of previously-learned materials by recalling facts, basic concepts & answers? #cmechat
  • dean_jenkins     T4 If you say someone is intended to 'be able to analyse something' then you need to assess their analytical skills. #cmechat
  • improvecme      @cmeadvocate you've never seen "Exhibit memory of previously-learned materials by recalling facts, terms, basic concepts & answers? #cmechat
  • dean_jenkins     Bloom's taxonomy is cognitive. Moore's outcomes framework is process-based. They overlap but aren't the same. #cmechat
  • CMEChat          T4: Say you were locked into your “SMART Bloom” objectives…how/when will you assess whether the objective was met? Be specific pls. #CMEchat
  • kenny_cox        Does Bloom go beyond moore level 4? RT @cmeadvocate: perhaps but does Moore's Levels 3-5 = Bloom's Taxonomy + SMART goal setting? #CMEchat
  • dean_jenkins     @cmeadvocate yes 'intended learning outcomes' = SMART objectives. They are measurable [with assessments]. #cmechat
  • cmeadvocate    Simplifying perhaps but does Moore's Levels 3-5 = Bloom's Taxonomy + SMART goal setting? #CMEchat
  • betsymeded      T2 - from my cs experience, I think there is a correlation, but I don't c many providers able to tie or try to tie these together. #CMEChat
  • theCMEguy       And avoid generic descriptors like "understand" RT @betsymeded: Use Action words to support needs => architechture => assessment. #CMEChat
  • cmeadvocate    @dean_jenkins - objectives written to be specific, measurable, action oriented...added complexity to Bloom's taxonomy, right? #cmechat
  • theCMEguy       @dgarbis CME = Continuing Medical Education #CMEchat
  • cmeadvocate    .@improvecme - do you think theses competencies exist in this 'profession' - never seen a cognitive domain driven program #CMEchat
  • dean_jenkins     T3 aren't 'smart' objectives 'intended learning outcomes'? I use 'aims', 'objectives' and 'outcomes' when planning. #cmechat
  • cmeadvocate    T3. Before you can create a SMART objective you would need to know quantitatively what is wrong & have hypotheses for solutions. #CMEchat
  • theCMEguy       T2: Some cross-pollination can exist tho, yes? Bllom's knowledge can affect change in competence, etc #CMEchat
  • dgarbis #cmechat Dennis Garbis from Newport Beach, CA here as curious visitor. What is the meaning/purpose of "CME"? TIA
  • spelletier           Pun of the day! RT @cmeadvocate: T2 Moore is trying to represent so much "moore" than learning - but in a very simple way. #cmechat
  • improvecme      T2 #CMEchat Yes, start w/needs, identify "domains" of needs, then identify objectives in those domains, then assessments in those domains.
  • betsymeded      T1 - Use Action words to support needs => architechture => assessment. ie. 'discuss' leads me to expect a discussion, beyond a Q&A #CMEChat
  • kenny_cox        RT @kenny_cox: RT @kenny_cox: Absolutely RT @cmeadvocate: so your needs would have to support each word, right? #cmechat
  • CMEChat          T3: SMART criteria may also drive LO's creation. What info is needed to craft of SMART objectives? #CMEchat t.co/zFCWRkR
  • dean_jenkins     T2 - getting very philosophical now. Bloom's taxonomy and Moore's pyramid (which could have more "dimensions"). #cmechat
  • theCMEguy       Yes RT @cmeadvocate: so your needs would have to support each word, right? RT @kenny_cox: Not a bad list #cmechat t.co/JOwhLvq
  • improvecme      #CMEChat T2 Clearly is not perfect fit, but Bloom's Knowledge works with Moore Level 3, Bloom Application may work with Moore's 4 subjective
  • LynchInsights    RT @dean_jenkins: +1 RT @cmeadvocate: the action words need to be [kept in] mind through out the whole planning process...#CMEchat #cmechat
  • cmeadvocate    so your needs would have to support each word, right? RT @kenny_cox: Not a bad list to keep around #cmechat t.co/jc4oYXQ
  • rmtyner T2 I think Bllom's and Moore's can be synergetic #CMEchat
  • dean_jenkins     +1 RT @cmeadvocate: the action words need to be [kept in] mind through out the whole planning process...#CMEchat #fingertips #cmechat
  • cmeadvocate    T2 - Blooms is about learning....Moore's is trying to represent so much "moore" than learning - but in a very simple way. #CMEchat
  • kenny_cox        Not a bad list to keep around for key words #cmechat t.co/kxcTmEl
  • spelletier           anyone have an answer? MT@CMEChat How do you relate Bloom’s work to Don Moore’s pyramid? #cmechat
  • cmeadvocate    T2 - Does the entirety of Blooms taxonomy fit into the 'learning' level of Moore's pyramid...is it that simple? #CMEchat
  • CMEChat          T2 - Does the entirety of Blooms taxonomy fit into the 'learning' level of Moore's pyramid...is it that simple? #CMEchat
  • theCMEguy       And I just realized somebody stole my list!!! Hey! RT @SusanBConnelly: Did that 2 when I used to write them #cmechat
  • CMEChat          T2: Bloom suggests levels of learning outcomes from remembering 3 creating - How do you relate Bloom’s work to Don Moore’s pyramid? #CMEchat
  • theCMEguy       I would agree MT @cmeadvocate: so it seems that the action words need to be mind thru out the whole planning process...#CMEchat #fingertips
  • SusanBConnelly            Did that 2 when I used to write them RT @theCMEguy: @SusanBConnelly Nice idea! I keep a list of action words taped beside my desk. #cmechat
  • cmeadvocate    T2: Bloom suggests levels of learning outcomes from remembering 2 creating - How do you relate Bloom’s work to Moore’s pyramid? #CMEchat
  • cmeadvocate    so it seems (could be wrong) that the action words need to be from of mind through out the whole planning process...#CMEchat #fingertips
  • cmeadvocate    so it seems 9could be wrong) that the action words need to be from of mind through out the whole planning process...#CMEchat #fingertips
  • dean_jenkins     Installing GNU Scientific Library and joining #CMEchat ... #multitasking
  • dean_jenkins     T1 - agree with @SusanBConnelly keeping a list of objectives' action words to handy is really useful. #cmechat
  • kenny_cox        Kenny from MO getting in late to #cmechat
  • cmeadvocate    backawards, no? RT @dean_jenkins: T1 - Usually the assessment (and level of the assessment) dictates the 'action' word in my view. #cmechat
  • cmeadvocate    q2 two partner ;-) RT @improvecme: T1: I try also to relate the categories in Bloom's taxonomy to Moore's Levels of outcomes. #CMEChat
  • theCMEguy       #CMEchat RT @theCMEguy: @SusanBConnelly Nice idea! I keep a list of action words taped beside my desk.
  • dean_jenkins     T1 - tough question. Usually the assessment (and the level of the assessment) dictates the 'action' word in my view. #cmechat
  • improvecme      T1: I try also to relate the categories in Bloom's taxonomy to Moore's Levels of outcomes. #CMEChat
  • cmeadvocate    T1 - what about the thoughts on remebering vs creating vs synthesis...different levels of cognition. Is this possible in CME #CMEchat
  • rmtyner T1 I also like LOs that build on each other #tellastory #CMEchat
  • Wyldceltic1       RT @doctorfunnybone: Survey: 45% of attendees said they'd like more jokes in presentations! Use ours anytime. Laughter IS the best medicine! #CMEchat
  • cmeadvocate    now that is funny! RT @theCMEguy: Wow - #CMEchat spam. We've hit the big time!
  • theCMEguy       Wow - #CMEchat spam. We've hit the big time!
  • SusanBConnelly            T1: I usually start by writing the rest of the objective then go back to find the action word that fits the desired outcome #CMEChat
  • cmeadvocate    T1 - I am not a great student of Bloom's Taxonomy...so I am interested in other perspectives...#CMEchat
  • theCMEguy       T1: Action word should relate to what you are trying to accomplish; what activity is designed to do or change #CMEchat
  • doctorfunnybone           Survey: 45% of attendees said they'd like more jokes in presentations! Use ours anytime. Laughter IS the best medicine! #CMEchat
  • dean_jenkins     Dean from UK, BMJ Group joining #cmechat
  • rmtyner T1 - my LOs tie back to the need/gap of the audience what am i trying to teach #CMEchat
  • spelletier           Sue just now joining in; will be in and out today #cmechat
  • cmeadvocate    T1 I love this topic...b/c I think the action word is so fundamental to following the linkages in educational planning...#CMEchat
  • CMEChat          T1: How do u choose the ‘action word’ in ur learning objectives? Thoughts on Bloom’s Taxonomy? t.co/IqJKQG2 #CMEchat
  • SusanBConnelly            Susan here in Lansdale PA, just getting off a call. #CMEChat
  • DownstateCME Edeline fm brooklyn here #cmechat
  • cmeadvocate    sounds like a small crew today....please shout out if you have just joined and we will get started...#CMEchat
  • cmeadvocate    @theCMEguy - b/c out of sight out of mind...over & over I watch planners & faculty forget about virtual learners...afterthought #CMEchat
  • cmeadvocate    I think we have a good topic today...maybe a little more dense than some weeks...so lets hope we can make it fit...#CMEchat
  • theCMEguy       @cmeadvocate Why so? #CMEchat
  • cmeadvocate    BTW, if you ever run a session where half attendees are live and half are online - be overly cautious of online learners...#CMEchat
  • improvecme      #cmechat Derek Dietze here — focused on outcomes measurement, swimming in data, working to improve CME.
  • LynchInsights    Tim here from NYC, multi-tasking today so I will be in and out. #cmechat
  • cmeadvocate    i should jot down notes from time to time as it relates to where the #CMEChat topics come from...this week comes from convo at #CMEchat
  • cmeadvocate    Brian behind the curtain here...already packed for #med2 but glad to be moderating #CMEchat...
  • theCMEguy       Derek here. Supplementing my #cmechat experience with a grape blow-pop
  • CMEChat          As with each #CMEchat the archive will be available at t.co/QfhE1UR and archived as a pdf on www.CMEadvocate.com
  • CMEChat          As with each #CMEchat we will guide the convo with 3 or 4 topics, but the real learning and teaching lies in your tweets - please engage.
  • CMEChat          Rule 7. A few mins before the end of #CMEchat, reintroduce yourself. Links, requests & shameless plugs welcome, too - #makingfriends
  • rmtyner Michelle - back home again in Indiana - always looking to expand my knowledge #CMEchat
  • CMEChat          Rule 6. On #CMEchat we aim to play nice. Sarcasm & irony welcome though. Really. Seriously. #laughingislearning
  • CMEChat          Rule 5. Periodically RT questions so those outside #CMEchat know what you’re talking about & so they can chime in too
  • CMEChat          Rule 4. Please remember to use the #CMEchat hashtag so all of the #CMEchat participants can follow the #CMEchat discussion!!!!
  • CMEChat          Rule 3. When writing, complete thoughts help followers outside chat learn from you. #CMEchat
  • CMEChat          Rule 2. [try to] stay on the #CMEchat topic. A new question will be asked every 12 min or so. If you can, include T# in related responses.
  • CMEChat          Rules for #CMEchat: 1. Introduce yourself. Location? Focus? Fave topics? What brings you here today?
  • CMEChat          Welcome to #CMEchat. All discussions herein are assumed to be personal opinion & not that of current, past, or future employers.
  • CSO_ImproveCME        @cmeadvocate Will do. Suggestion 4 future chat: Adapting "Strong Inference" discipline (Science, 1964; 146:347) to CME science. #cmechat
  • LynchInsights    RT @cmeadvocate: Today we will be discussing Bloom’s Taxonomy, here are 2 resources: t.co/ESNH3kc & t.co/UonVUMK #cmechat
  • profragsdale     RT @cmeadvocate: Today we will be discussing Bloom’s Taxonomy, here are 2 resources: t.co/A3VUAkb & t.co/jfybBgg #CMEchat
  • cmeadvocate    @CSO_ImproveCME - we will miss you...but feel free to reconnect with the archive and continue the conversation later on... #cmechat
  • CSO_ImproveCME        @cmeadvocate Hey Brian. Will not be able to join today's chat. Too many deadlines. Until next time...#cmechat
  • cmeadvocate    lets see where it fits...RT @improvecme: #cmechat would like to also discuss linking objectives and assessments to Moore's Levels.
  • cmeadvocate    Today we will be discussing Bloom’s Taxonomy, here are 2 resources: t.co/A3VUAkb & t.co/jfybBgg #CMEchat
  • improvecme      #cmechat having objectives by Moore Levels with corresponding content and assessments.
  • improvecme      #cmechat would like to also discuss linking objectives and assessments to Moore's Levels.
  • cmeadvocate    60 MINUTES to #CMEchat - Linking learning objectives and assessment methods. Join us at 11AMET
  • spelletier           RT @cmeadvocate: Can CME save lives? The results of a Swedish, evidence-based CE intervention. t.co/VKlt6dz #CMEchat [pdf] #cmechat
  • CuratioCME      MT @cmeadvocate: Just over 3 hours to #CMEchat - Linking learning objectives and assessment methods. Join us at 11AMET
  • theCMEguy       MT @cmeadvocate: Just over 3 hours to #CMEchat - Linking learning objectives and assessment methods. Join us at 11AMET
  • cmeadvocate    Just over 3 hours to #CMEchat - Linking learning objectives (exploring Bloom's & SMART framework) and assessment methods. Join us at 11AMET

Posted by Brian S McGowan PhD at 9/14/2011 5:12 PM
Categories: uncategorized
 

What did you think of this article?




Trackbacks
Trackback specific URL for this post
  • No trackbacks exist for this post.
Comments
  • No comments exist for this post.
Leave a comment

Submitted comments are subject to moderation before being displayed.

 Name (required)

 Email (will not be published) (required)

 Website

Your comment is 0 characters limited to 3000 characters.