The Blog
Evaluation, research, workforce development, and nonprofit leadership — from Rojas Blakely & Associates.
-
ArticleWhy we invested in professional data visualization (and why it matters for our clients)
April 28, 2025
When we launched our new brand, we took the opportunity to go beyond choosing our colors or logo. We wanted to make sure that everything we created consistently reflected who we are: a firm committed...
Read more → -
ArticleLeadership Insights from Kamala Harris: What we can learn from the presidential campaign.
December 10, 2024
Even though Kamala Harris lost the presidential election, she exemplified great leadership traits during the campaign. We thought we would share them with emerging, new, and seasoned...
Read more → -
ArticleNew Labor Market Study Shows Gaps Between Employers and Workforce Training System
November 29, 2023
Opportunities Exist in Middle-Skill Jobs that Do Not Require College Degrees Link to web versions at cfgnh.org and valleycommunityfoundation.org New Haven, CT (Nov. 3, 2023) — As job openings across...
Read more → -
ArticleEvaluation quick start
November 24, 2022
The year is almost over but it's not too late to create the most effective evaluation plan your program has ever had. In fact, now is the perfect time to use evaluation to reflect back on the past...
Read more → -
Article
3 takeaways from talking data viz and racial equity on the Policy Viz Podcast
November 23, 2022
A few weeks ago, Eli Holder and I were invited to appear on the Policy Viz Podcast with Jon Schwabish. We were following up on our article on why it's important to consider how you're presenting data...
Read more → -
ArticleGraduation keynote for YMCA Training Inc
August 25, 2022
I was asked to deliver a keynote address at the graduation of YMCA Training Inc's 132nd cycle. The YMCA Training, Inc. program is an intensive 20-week, full-time computerized office skills training...
Read more → -
Article2 Things That Need To Be In Place Before You Can Build An Evaluation System
August 15, 2022
This post first appeared on the Greater Boston Evaluation Network site in 2018 A full evaluation program requires data systems and analytic capacity. Many organizations or programs feel like they’re...
Read more → -
ArticleWhat can go wrong? Racial equity, dataviz, & deficit thinking
July 22, 2022
In 2019, Pieta published a blog post on data visualization that was consistent with targeted universalism. Jonathan Schwabish and Alice Feng at Urban Institute picked up on that post and cited it in...
Read more → -
ArticleBuilding feedback loops, one metric at a time
May 12, 2022
One of the tools that I use to manage my clients’ evaluations is Social Suite. It’s a powerful online tool that makes it possible to manage complicated evaluations and turn around analysis fast. I...
Read more → -
ArticleMeasuring quality, quantity, and results
May 2, 2022
One of the risks when developing an evaluation plan, is developing measures without being really clear about what kind of measure they are or how we're going to use them to understand our program. In...
Read more → -
ArticleEvaluation FAQs
March 15, 2022
Managing your evaluation plans internally can be complicated. Trying to design the evaluation plan that best represents your work without disrupting the work is crucial and challenging. There are a...
Read more → -
ArticleAnniversary show with Intentionally Act
April 22, 2021
Over the past year, we have discussed Covid and it's interaction with racial equity, disaster recovery, and building resilient communities. Who better to talk about these themes than Dr. Atiya...
Read more → -
Article
Measuring SES series: Income, need, and living wages
January 5, 2021
A couple of weeks ago, I wrote a post on socio-economic status (SES). SES is usually understood to be a combination of three things: income, education, and occupation. There are multiple reasons why...
Read more → -
Article
Measuring socio-economic status
November 19, 2020
A couple of weeks ago, I wrote a blog post on gathering race and ethnicity data on a survey or form. One of my readers commented that socio-economic status (SES) was also important and related. A...
Read more → -
Article
Race and ethnicity: Collecting demographic data in a survey
October 21, 2020
There are multiple reasons why you might want to collect race/ethnic data on a survey and each of those reasons will have different implications for what you ask and how. In this post, I will clarify...
Read more → -
Article
KPIs are for driving performance: Four questions to ask about your program's KPI
April 23, 2020
To evaluate performance, most organizations select one or two KPI(s) — Key Performance Indicator(s) —to focus on. A KPI is the outcome measure that your organization is most focused on. But once...
Read more → -
Article
Selecting a Key Performance Indicator
April 14, 2020
Author’s note: this post borrows from the Results-Based Accountability Framework as well as Trying Hard is Not Good Enough by Mark Friedman. What is a KPI? KPI stands for Key Performance Indicator....
Read more → -
Article
Data collection in the time of COVID-19
March 24, 2020
Well, these are unprecedented times. Last week, I held a Facebook Live to talk about some of the ways that programs might be adjusting their data-collection and use right now as we grapple with...
Read more → -
ArticlePredicting the future: How to make a projection or a target
March 9, 2020
In a previous post, I discussed how to chart imputed, real, and projected data - and I promised to discuss in a future post how to project future data. I keep my promises, so here it is: how to make...
Read more → -
Article
Charting imputed, real, and projected data
January 30, 2020
In a previous post, I discussed the importance of keeping a data agenda: a list of data that will be helpful to your program but is not currently being measured. Once you’ve completed your agenda,...
Read more → -
Article
Keeping a data agenda
December 19, 2019
Previously, I discussed the importance and benefits of keeping a data dictionary. But as your program grows, you’re going to want to get familiar with the data dictionary’s close cousin - the...
Read more → -
Article
Keeping a data dictionary
November 20, 2019
Program evaluation is a subject that often causes angst among nonprofit leaders. Though the leaders in your organization may agree it is important to track the efficacy of your programs, the...
Read more → -
Article
Using a data walk to redesign a workforce development training program
November 6, 2019
Are you and your team trying to make better sense of your data? You might want to consider a Data Walk. A data walk is a simple exercise that helps staff and stakeholders understand their data and...
Read more → -
Article
Does sustaining your organization go in your logic model?
September 18, 2019
Every once in a while, I see a logic model that incorporates the organization's financial sustainability. The rationale for it is that if the organization doesn't have enough resources, the program...
Read more → -
Article
Sex, gender, and sexual orientation: Some resources on collecting demographic data in a survey
March 20, 2019
A few weeks ago, I published a tip on asking about gender on a survey. The subject obviously requires a lot more than one tip. Below are several excellent resources on asking about gender or sexual...
Read more → -
Article
More useful guides and resources for building logic models
March 13, 2019
Last year, I published a post with a handful of resources on developing logic models. It's a new year, and I thought the subject was worth revisiting. This post includes more long-form guides and...
Read more → -
Article
Presenting data for a Targeted Universalist approach
March 6, 2019
Targeted universalism is an approach that emphasizes reaching a population-level goal by ensuring that each of our sub-populations meets the goal. I’ve been thinking about the implications that...
Read more → -
Article
Aligning logic models with Targeted Universalism
February 27, 2019
I have recently been thinking and reading about Targeted Universalism. Originally developed by professor and critical race scholar John A. Powell, Targeted Universalism is an approach to advancing...
Read more → -
Article
Types of evaluation studies: Process, outcome, and impact
February 13, 2019
I have written in other posts about creating a program logic model and incorporating measurement into a logic model. {{cta('43979b85-ab72-49bc-a0b0-44a90d372251','justifycenter')}} One of the reasons...
Read more → -
Article
Ensuring a diverse sample without asking for sensitive information
December 29, 2018
I recently had a project where I was interviewing participants at a program that serves a very diverse group. I knew from talking with the program managers, that the population they served was...
Read more → -
Article
Types of performance measures
November 28, 2018
In this post, I’m talking about how to write good performance measures. First, a clarification. I use the term performance measures to include all the output and outcome measures that can be used in...
Read more → -
Article
A close relationship with staff as a measure of program quality
November 12, 2018
It is important in many programs that the participants form close and trusting relationships with staff or volunteers. In mentoring, in particular, the formation of a trusting relationship is crucial...
Read more → -
Article
A large sample of the wrong people is just a waste of time: How to tell if your sample is representative
October 3, 2018
I recently posted a #tiptuesday in which I said that a sample of the wrong people was a waste of time. A friend asked “how do you know it’s the wrong people?” That’s a great question and I’ll address...
Read more → -
Article
Working with youth advisers
September 10, 2018
I always enjoy interviewing youth. I find them insightful, frank, smart, and funny. But working with youth advisers to plan interviews with youth participants can make the interview process go even...
Read more → -
Article
Questions and answers on focus groups
July 11, 2018
I recently had the opportunity to present a workshop on running focus groups for a group of parent advocates. They were preparing to hold focus groups with parents to learn about their engagement...
Read more → -
Article
When the intervention affects the entire population: Interrupted Time Series
May 2, 2018
Most program evaluations depend on comparing people who did participate in our program to people who didn't. But how can we measure the effects of a program that affects everyone in a population? How...
Read more → -
Article
Better survey results: Presenting the results of a retrospective pre-post design
April 5, 2018
A retrospective pre-post design is one where you ask learners to evaluate their own learning after a training event. This is a great alternative to administering two surveys, one before...
Read more → -
Article
Selecting a logic model template: Some useful (and very readable) guides to building logic models
March 6, 2018
The first thing to know about logic models and logic model guides is there is no single righ way to build a logic model. You'll want to try out a few templates and guides and see what works best for...
Read more → -
Article
On writing good performance measures: Who cares?
March 6, 2018
Sometimes, when I’m working with staff to document their program’s performance measures, I notice that they come up with some very specific descriptions of what it takes to run a quality program. For...
Read more → -
Article
What internal evaluators do: Formative evaluation
March 5, 2018
One of the roles that the internal evaluation team or measurement, evaluation, and learning (MEL) plays is to gather data on ongoing programs. They provide this data to program managers so that they...
Read more → -
Article
I’ve been gathering data in the field. How do I know when I’m done?
February 8, 2018
Professional program evaluators always start a study with a sampling plan. But program managers and other professionals often find themselves gathering data in a less structured way. We often need to...
Read more → -
Article
Better survey results: Writing great questions
February 5, 2018
Surveys are a go-to method in program evaluation, used both to collect feedback from individuals who have participated in our programs and also to understand the needs and opinions of people in the...
Read more → -
Article
The difference between strategic management and process improvement
January 26, 2018
In my blog posts, I talk a lot about various types of performance measures, how to develop and collect them, and how to incorporate them in a logic model. One thing that I have been thinking a lot...
Read more → -
Article
What administrative data is and why you should use it
January 22, 2018
When we are setting out to evaluate or report on our programs, we often assume that we have to create new data, generally by going back to our participants and having them complete surveys,...
Read more → -
Article
Using research and evaluations to complement your internal evaluation
January 3, 2018
In previous posts, I've mentioned that the further outcomes are in time, the harder it is to collect that outcome data. For example, if your program provides tutoring services to middle...
Read more → -
Article
3 keys to selecting measures for nonprofit performance management
December 18, 2017
Many nonprofits are looking at performance management as a way to improve their ability to provide services and to use data intentionally for strategic planning. Here, when I say performance...
Read more → -
Article
The difference between process indicators and performance measures
December 13, 2017
Many organizations struggle with writing their own performance measures. The goal is to create performance measures that describe the program's impact well and are useful for strategic management....
Read more → -
Article
Including performance measures in a logic model for program evaluation
December 8, 2017
Now that you have developed a logic model (check out this post if you still need some help, or download my logic model guide), you might be wondering how to integrate your data collection plans with...
Read more → -
Article
Demographic data on a survey: Why to collect it and what to do with it
December 6, 2017
When we're conducting a survey for program evaluation, we have probably all been told to collect some basic demographic data on every survey, but what data and why? The most important thing you can...
Read more → -
Article
Implementing surveys in program evaluation: Informed consent
November 30, 2017
In program evaluation, we often don't have to go through and IRB or ethics review before we conduct a survey. That means we are responsible for conducting an ethical survey ourselves. The consent...
Read more → -
Article
Some useful (and very readable) guides to building logic models
November 15, 2017
The basis of an internal evaluation program for a nonprofit organization is clear logic models. The first thing to know about logic models is there is no single right template. You'll want to try out...
Read more → -
Article
Better survey results: Sampling
November 6, 2017
Whether its for a research project or a program evaluation, one of the hardest things about implementing a survey is understand who you should have respond to it. A survey is only a useful method of...
Read more → -
Article
Better survey results: Likert scales (or ratings scales) and how to interpret them
November 2, 2017
A very common and useful question type in survey research is the Likert scale. That’s the kind that asks the respondent to agree or disagree with a statement, or rate the extent to which they liked...
Read more → -
Article
Demographic data on a survey: Why to collect it and what to do with it
October 25, 2017
We have probably all been told to collect some basic demographic data on every survey, but what data and why? The most important thing you can do with your demographic data is to demonstrate that the...
Read more →