• T1: We’ve discussed needs, objectives, & interventions before; what are the major challenges to collecting ‘O’utcomes data #CMEchat
• T2: Many providers think of outcomes as “pre/post” or “30/60/90 day” follow-up? Are there other times to ‘assess’ outcomes? #CMEchat
• T3: Some argue that real learning only occurs with feedback, how do you tie assessment and feedback together to support learning? #CMEchat
• T4: What are the best practices/resources for building assessment plans? #CMEchat
• T5: Moore et al proposed a complex conceptual framework for CME (see: yfrog.com/mfcbdp) How have you used this in practice? #CMEchat
• T6: CME planning tables are increasingly used to support planning. What are pros/cons of such a tool? #CMEchat (see: bit.ly/o0Ze9a )

You can see the discussion below or download the transcripts HERE:

• CMEChat          The real impact of the #CMEchat lies in your action - engage others to participate & please take action on things your have learned! G’day!
• CMEChat          As we get wrap up, please vote for the tweet of the day (#TOTD) seen during this week's #CMEchat.
• CMEChat          Thx for participating in #CMEchat. All discussions herein are assumed 2 b personal opinion & not that of current, past, or future employers
• cmeadvocate    ok - looks like an abrupt ending to the #CMEchat with the #failwhale...please continue the conversation if the technology allows...
• meducate          @cmeadvocate #CMEChat What? Did you say something?
• cmeadvocate    If a tree falls in the woods...does anyone hear it fall? #CMEchat
• cmeadvocate    T5: Moore proposed a complex conceptual framework for CME - try this one: hhttp://bit.ly/rfloEQ #CMEchat
• cmeadvocate    Looks like twitter has gotten really rrreeeaaaallllllllyyyyyyyyyy slow right now...more time for reflection! #CMEchat
• CMEChat          Here is another link to Moore's Conceptual model: t.co/W5UvJpZz #CMEchat
• CMEChat          T5: Moore proposed a complex conceptual framework for CME (see: t.co/W5UvJpZz). Is this practical in today's CME? #CMEchat
• cmeadvocate    t4 - sensing some lack of engagement today on #CMEchat too ;-) Do we not have answers to these questions or are we throttled by twitter too?
• cmeadvocate    t4: the ARRAY model was used at #AMACME & PollEverywhere was used at #ACEPmtng. Others use iPads. But does this help? #CMEchat #shinyobject?
• meducate          T4 #CMEChat Technology should be a powerful help in data collection
• theCMEguy       The glories of working from home: 4 y/o demanding lunch. I'm off #cmechat. Thanks for convo!
• cmeadvocate    t4: we have heard about fatigue, engagement, simplicity (happy medium) - how about actual modes of collection? can technology help? #CMEchat
• theCMEguy       You mean like we had at #amacme? MT @cmeadvocate: ...but not like those awkward "time to reflect" moments. #CMEchat
• cmeadvocate    gentle correction: RT @theCMEguy: @cmeadvocate Exactly, you [MUST] combine feedback and assessment #CMEchat
• theCMEguy       @cmeadvocate Exactly, you can combine feedback and assessment #CMEchat
• CMEChat          T4: What are the best practices/resources for building assessment plans? #CMEchat >>seems like we have a few ideas in play already...
• cmeadvocate    I've always believed that assessment MUST be integrated into the learning...but not like those awkward "time to reflect" moments. #CMEchat
• cmeadvocate    so FEEDBACK can be crowdsourced! RT @theCMEguy: T3: This is why I love the idea of "turning on" comment boxes for CME activities. #cmechat
• cmeadvocate    so assessment can be crowdsourced! RT @theCMEguy: T3: This is why I love the idea of "turning on" comment boxes for CME activities. #cmechat
• cmeadvocate    @edCetraT - I agree that looking bkwrd offers valuable lessons (Olson's work on serendipity) but how does it relate to planning  #cmechat
• meducate          @edCetraT #CMEchat - thanks for your participation!
• edCetraT           short but sweet - gotta run - thank you #cmechat
• theCMEguy       T3: This is why I love the idea of "turning on" comment boxes for CME activities. #cmechat
• meducate          @cmeadvocate T3 #CMEChat - Exactly!  Feedback is based on interim assessment.  And assessment is ongoing, not episodic!
• akroundtree       And other ways besides surveys? RT @CMEChat: T2 Are there other/better times to ‘assess’ outcomes? #CMEchat
• cmeadvocate    T3 - the point being - how can you provide FEEDBACK if you are not providing FORMATIVE assessment...right? #CMEchat
• edCetraT           @cmeadvocate so instead of the outcome of taking this training is..lets look at what happened during the training... #cmechat
• theCMEguy       I'd agree with that assessment RT @cmeadvocate: t2 - I'd bet that fewer than 1:10,000 objectives are written with a time component. #CMEchat
• cmeadvocate    look who has exactly 7,000 followers! RT @Doctor_V: I'm in clinic. Will lurk #CMEchat
• meducate          Welcome none the less! RT @Doctor_V: I'm in clinic. Will lurk #CMEchat
• Doctor_V          I'm in clinic. Will lurk #CMEchat
• CMEChat          T3: Some argue that real learning only occurs w/ feedback, how do you tie assessment & feedback together to support learning? #CMEchat
• meducate          @theCMEguy #CMEChat - it can be
• edCetraT           @cmeadvocate #projecttincan coming from the adl is a example of pushing 'analytics' to incorporate collection of 'experiences' #cmechat
• cmeadvocate    t2 - I'd bet that fewer than 1:10,000 objectives are written with a time component...so long-term outcomes are pointless, right? #CMEchat
• theCMEguy       Is that always bad? RT @meducate: T1/2 #CMEChat ...many try to measure beyond what they are teaching
• meducate          T1/2 #CMEChat Just remember that outcomes are based on objectives - and many try to measure beyond what they are teaching
• cmeadvocate    @edCetraT  - what do you suggest instead? ( like where you are going with this...) #cmechat
• akroundtree       Tips for long-term? RT @CommPharm: T1 short-term v long-term outcomes.  Short-term might be easier to measure versus long-term  #cmechat
• edCetraT           "T2 - Assessments can also 'measure' or provide data on the experience itself - much like what web analytics do for web sites
• #cmechat"
• cmeadvocate    they try to...RT @theCMEguy: T2: fair to say that many CME pros assess on a templated schedule? #cmechat
• cmeadvocate    T2 - i think the inportance of on-going or formative assessment SO critical...but so rarely done or reported...ideas? #CMEchat
• edCetraT           T2 - I'm going to say that focusing solely on measuring 'Outcomes' is a big mistake and a reason why we do the pre-ppost, etc. #cmechat
• theCMEguy       T2: fair to say that many CME pros assess on a templated schedule? #cmechat
• rmtyner So many think they know everything... how do you start that conversation?? #CMEchat RT @meducate: T 1/2 #CMECh… (cont) t.co/FTCdwpAl
• cmeadvocate    T1/2 - for example...when providing skills training...the hands on process makes real-time assessment much more 'tangible', no? #CMEchat
• meducate          T 1/2 #CMEChat I also think that the lack of traniing of CME professionals in assessment leads to poor strategy and design
• theCMEguy       T2: Ongoing assessment would be great, especially if you have curriculumn in place rather than just o"one-offs" #cmechat
• cmeadvocate    t1/2 so if training and education was taken from the class room to the practice wouldn't assessment be much simpler? #CMEchat
• cmeadvocate    no judging here...no dumb comments....love to hear from the new members...what are your barriers to assessment? #CMEchat
• edCetraT           T1 - Many organizations run training as a separate organization within an organization and don't link data from training to bizness #cmechat
• CMEChat          T2: Many providers think of outcomes as “pre/post” or “30/60/90 day” follow-up? Are there other/better times to ‘assess’ outcomes? #CMEchat
• meducate          T1 #CMEChat - speaks to the value of ongong formative evaluations so not just waiting for summative evaluation
• cmeadvocate    "if i would have more time I could have written you a shorter letter..." #CMEchat
• cmeadvocate    t1 - can planners share responsibility with learners to build accountability/engagement in outcomes. #CMEchat
• cmeadvocate    T1 - THESE are OUR objectives....and THIS is how WE will evaluate our successes? - new opening to all programs? #CMEchat
• theCMEguy       T1: if it's too complex, no one will do it. If too simple, get useless data. #happymedium #CMEchat
• akroundtree       Good point. RT @theCMEguy: T1: On a very basic level, a big challenge is simply getting people to complete suveys/forms/etc #cmechat
• meducate          T1 #CMEChat - it is east to make a complicated evaluation, it is difficult to make a simple and effective one
• CommPharm     T1 - there is a both short-term and long-term outcomes.  Short-term might be much easier to measure versus long-term  #cmechat
• cmeadvocate    t1. if assessments were simplified (answered fewer questions) but more meaningful responses - is this better? #CMEchat
• meducate          T1 #CMEChat Know what to ask, how to ask it, who to encourage to complete it, and explain why we need it
• cmeadvocate    Ok  Fatigue - yes. unawareness - yes. How about complexity vs simplicity? #CMEchat
• rmtyner Agree! Needs to be easier for the learners #CMEchat RT @meducate: T` #CMEChat had this discussion yesterday wi… (cont) t.co/yNHKn0ct
• theCMEguy       T1: Right, not just completing them, but doing so in a meaningful fashion. #cmechat
• meducate          T` #CMEChat had this discussion yesterday with some folks at #rcpscpd - there is outcomes fatigue plus no understanding of why its important
• cmeadvocate    @theCMEguy - i like that...on a basic level....I think we will will find levels of complexity herein... #cmechat
• cmeadvocate    t1 - we have all heard about and maybe even experienced first hands Docs and outcomes fatigue...#CMEchat
• theCMEguy       T1: On a very basic level, a big challenge is simply getting people to complete suveys/forms/etc #cmechat
• cmeadvocate    outcomes assessments OR evaluations...we may need a glossary! RT @meducate: T1 #CMEChat is that an uppercase O in Outcomes?
• meducate          T1 #CMEChat is that an uppercase O in Outcomes?
• CMEChat          T1: We’ve discussed needs, objectives, & interventions before; what are the major challenges to collecting ‘O’utcomes data #CMEchat
• cmeadvocate    19 left! RT @theCMEguy: Let me guess...is it about golf? RT @cmeadvocate: BTW - I think I am writing a book! #CMEchat
• cmeadvocate    to Rob and Aimee - thanks for joining us! #CMEchat
• meducate          It was beautiful in Ottawa this week! #cmechat
• theCMEguy       Let me guess...is it about golf? RT @cmeadvocate: BTW - I think I am writing a book...but that's all I can say right now! #CMEchat
• cmeadvocate    oh Canada! RT @CommPharm: Hi Mike from Barrie, Ontario. Beautiful here as well. #CMEchat
• akroundtree       #cmechat Aimee, medical writer, CME materials, modules developer
• CommPharm     Hi Mike from Barrie, Ontario. Beautiful here as well. #CMEchat
• meducate          Welcome @rescgro #cmechat
• cmeadvocate    #CMEchat RT @resgro: Rob Esgro here, new to the CMECHAT but heard a lot about it, looking forward to learning from the experts
• cmeadvocate    BTW - I think I am writing a book...but that's all I can say right now! #CMEchat
• edCetraT           Hello! Nancy joining in from Toronto. @MetrixGroup join us! #cmechat
• meducate          @cmeadvocate Good idea #cmechat #ihearcrickets
• cmeadvocate    lets give folks a few minutes to join in...and see if our outreach #AMACME #rcpscpd AND #ACPEmtng helpd promote #CMEchat
• theCMEguy       I'm sitting in front of the AC because it still feels like a sauna here #wheresfall? #cmechat
• meducate          I've been working on arranging a special guest tweeter fro an upcoming #cmechat session...
• rmtyner Same here RT @spelletier: Sue here, with a brain still sore from last week's info dump at the Task Force meeting #cmechat
• cmeadvocate    Could it be that everyone is home this week? so much early fall traveling...#CMEchat
• meducate          @cmeadvocate That's how I opened my plenary! #rcpscpd #cmechat
• spelletier           Sue here, with a brain still sore from last week's info dump at the Task Force meeting #cmechat
• cmeadvocate    so true so true! RT @meducate: #CMEChat  Lawrence here, joining from NY having just returned from the home of CME research - Canada
• rmtyner This is Michelle... back home again in Indiana and ready to discuss CME assessments #CMEchat
• cmeadvocate    I am watching the launch of the new Kindle Fire - NEW TOYS ROCK! #CMEchat
• meducate          #CMEChat  Lawrence here, joining from NY having just returned from the home of CME research - Canada
• cmeadvocate    Thx for joining today and I look fwd to learning and sharing from you - i'm Brian and I will be moderating today #CMEchat
• theCMEguy       Derek here, coming to you live from home in Chestnut Hill #CMEchat
• CMEChat          As with each #CMEchat the archive will be available at t.co/nOLVEvIB and archived as a pdf on www.CMEadvocate.com
• CMEChat          As with each #CMEchat we will guide the convo with 3 or 4 topics, but the real learning and teaching lies in your tweets - please engage.
• CMEChat          Rule 7. A few mins before the end of #CMEchat, reintroduce yourself. Links, requests & shameless plugs welcome, too - #makingfriends
• CMEChat          Rule 6. On #CMEchat we aim to play nice. Sarcasm & irony welcome though. Really. Seriously. #laughingislearning
• CMEChat          Rule 5. Periodically RT questions so those outside #CMEchat know what you’re talking about & so they can chime in too
• CMEChat          Rule 4. Please remember to use the #CMEchat hashtag so all of the #CMEchat participants can follow the #CMEchat discussion!!!!
• CMEChat          Rule 3. When writing, complete thoughts help followers outside chat learn from you. #CMEchat
• CMEChat          Rule 2. [try to] stay on the #CMEchat topic. A new question will be asked every 12 min or so. If you can, include T# in related responses.
• CMEChat          Rules for #CMEchat: 1. Introduce yourself. Location? Focus? Fave topics? What brings you here today?
• CMEChat          Welcome to #CMEchat. All discussions herein are assumed to be personal opinion & not that of current, past, or future employers.
• cmeadvocate    In prep for #CMEchat please take a look at the Moore's Conceptual Framework on Assessment - t.co/S09bchU9 (beyond the pyramid)
• meducate          "RT @spelletier: The thrills, spills, and chills of CME assessments—#CMEchat will tackle it all in just 10 minutes. C'mon down!   #cmechat"
• spelletier           "The thrills, spills, and chills of CME assessments—#CMEchat will tackle it all in just 10 minutes. C'mon down!
• #cmechat"
• CuratioCME      MT @cmeadvocate: 15 mins until #CMEchat - we come clean about the challenges and share best practices with CME assessments - pls join us!
• theCMEguy       MT @cmeadvocate: 15 mins until #CMEchat - we come clean about the challenges and share best practices with CME assessments - pls join us!
• rmtyner MT @cmeadvocate: 30 mins until #CMEchat - we come clean about the challenges and share best practices with CME assessments - pls join us!
• cmeadvocate    30 mins until #CMEchat - today we come clean about the challenges and share best practices with CME assessments - pls join us!
• meducate          Let's see if we get any participants from north of the border on #CMEChat today #rcpscpd cc @RoyalCollegeCPD
• cmeadvocate    but if the overwhelmed Docs just got more CME.... RT @KentBottles: Study finds need for Docs in NE Phila. t.co/yUE4Jlgq  #CMEchat
• cmeadvocate    A report of quality improvement in the care of patients w/ acute coronary syndromes - t.co/yukAysXQ #CMEchat (mar2010) @MedIQCME
• cmeadvocate    @aafp: 31% said they were more likely to change behavior after an interactive session (vs 27% lecture) - t.co/UO3Y9Xdr #CMEchat
• cmeadvocate    @aafp: interactive (39%) & procedural (27%) formats were thought to be more effective than lecture (24%) - t.co/UO3Y9Xdr #CMEchat
• cmeadvocate    (46%) of @aafp respondents indicated that they prefer lecture-based format for CME activities - t.co/UO3Y9Xdr - #CMEchat
• cmeadvocate    "Social skills needed at med schools" - YES...a social learning culture is fundamental to future medicine...http://t.co/C66IIXgj #CMEchat
• cmeadvocate    PCPI: "..metrics 'must be clinically relevant, meaningful, measurable & actionable by the clinician' " (NICE): t.co/MBX9zUeE #CMEchat
• cmeadvocate    Engaging with the CME Environment - Both Nationally and Locally - ACCME [VIDEO]  t.co/nkctFVj9 #CMEchat (Oct 2010)
• cmeadvocate    @AAMCtoday annual meeting will have a "CE-PI" track: t.co/l7MClu9a #CMEchat Nov 5-8th in Denver, Co
• cmeadvocate    How dated does this blog from 2006 sound nowadays: "Who needs medical knowledge, anyway?" - t.co/XL0tgpCv #CMEchat #denialstage
• cmeadvocate    2006: Musings on continuing medical education background and foreground learning >> not sure i agree. t.co/NdwMWdve #CMEchat
• cmeadvocate    In one study, over 90% of users thought UpToDate was “integral” to their decision making. t.co/M8TMTZys #CMEchat
• CatalystRanch   great article about key opinion leaders! (#KOL) from @spelletier ! t.co/rIXvKD5M #CMEchat
• MoFehr Tweeting connects Learners across timezones #cmechat
• CuratioCME      RT @CMEChat: Pls join us tomorrow for #CMEchat 11AMET. TOPIC: Formative & Summative Assessment in CME - Bldg a Rapid-learning CME community.
• theCMEguy       RT @CMEChat: Pls join us tomorrow for #CMEchat 11AMET. TOPIC: Formative & Summative Assessment in CME - Bldg a Rapid-learning CME community.
• cmeadvocate    Pls join us tomorrow for #CMEchat 11AMET. TOPIC: Formative and Summative Assessment in CME - Bldg a Rapid-learning CME community. #rcpscpd
• CMEChat          Pls join us tomorrow for #CMEchat 11AMET. TOPIC: Formative and Summative Assessment in CME - Bldg a Rapid-learning CME community. #rcpscpd
• theCMEguy       So we can all expect a bunch of new followers and #CMEchat-ers? MT @meducate: About to start my SoMe in CME workshop... #rcpscpd
• cmeadvocate    @meducate - tell him he is one of my top picks...if he ever comes to PA? (unless you can guide him tomorrow?) #CMEChat
• meducate          @theCMEguy I raised the idea of guest tweeting on #CMEChat @cmeadvocate
• cmeadvocate    Key challenges in simulated patient programs: An international comparative case study - t.co/64dfSbBG #meded #CMEchat
• cmeadvocate    Nash's vision for the evolution of CME & CPD - the future of quality. t.co/L19EZUrD #CMEchat t.co/8ozEIl1o