2021

Impact Report 🎉

What is Generation Girl?

Generation Girl is a community established by Yayasan Generasi MajuBerkarya that equips young women with hard and soft skills to becomefuture female leaders through STEM-based programs. We host four coreprograms to teach our community five foundational skills: problem solving,critical thinking, confidence, communication, and compassion. Started in2018, Generation Girl has reached over 20,000 girls aged 12 onwards throughour programs, events, and workshops and hope to continue growing ourcommunity across Indonesia. Head over to our website for more information!

Summer Club In a Nutshell

Summer Club is a free, project-based program run during school holidays for girls in grades 9 to 12. We teach differentSTEM topics and sprinkle in speaker sessions and soft skills workshops to create a well-rounded program. Aside from improving the foundational skills of the participants, who we call Rookies, we also introduce them to technical fields they could pursue in their future careers. As we rotate topics weekly, Rookies present a final project everyFriday they worked on throughout the week that shows how much they learned in that topic. Our expectations for these projects are very high, but the Rookies always seem to exceed them!

This year, we taught Computer Science101, UI/UX Design, Game Design, DataScience, and Build a Website throughout the three weeks of Summer Club. Each topic has an English and Bahasa Indonesian class and is taught by our incredible volunteer Mentors andTeaching Assistants (TAs) who are trained by Generation Girl’s internal team. Twice a week, we also invite inspirational women to speak about different themes ranging from mental health to their unconventional career paths to widen theRookies’ network of role models. This year, we added two extra speaker sessions for a total of 8 sessions throughout the three weeks.

Aside from the 231 Rookies who participated in this year’s program, 24 Mentors, 18 TAs, 9 operational volunteers, 9 speakers, and 14 sponsors and partners helped us make SummerClub 2021 a reality.

This is our third-ever Summer Club and post-program feedback has always been very positive, but we wanted to understand the impact this program has on our Rookies. How can we improve it and make it even more impactful? Does it help us achieve our larger goal of creating more female leaders in Indonesia?

How did we measure impact?

Impact is really tough to measure, as it’s such an abstract concept (and this was our first time attempting to dissect it). We researched proven methods industry leaders used to analyze impact and modeled our impact framework off those. We decided to use pre-post testing to analyze the quantifiable outputs and outcomes and hope to do another survey after 12 months to measure long-term impact.

Summer Club has three different objectives:

Objective 1

Increase awareness of hard and soft skills through STEM subjects

Objective 2

Create support system and safe place for young women to learn

Objective 3

Strengthen Generation Girl’s relationship with industry leaders to increase brand awareness and network

Does our curriculum increase the Rookies’ hard and soft skills? Is Summer Club a space they feel comfortable exploring and learning in? Is the program helping us grow our network with industry leaders? We specified indicators for each objective to answer those questions and measure whetherSummer Club’s activities actually achieve its objectives.

We started by using previous event data as well as outside research to find

baselines for each indicator. These baselines were important, as it gave us a benchmark to understand the data we collected throughout this program.However, as this was the first program we actively measured impact for, some benchmarks are either missing or not a direct comparison to the findings fromSummer Club 2021. As we continue using this framework for future programs, this issue will hopefully be resolved in the future!

Here's what we found

Objective 1

Increase awareness of hard and soft-skills through STEM subjects

Summer Club has three different objectives:

For Objective 1, we measured whether this program had an effect on the Rookies’ foundational skills.

Although Generation Girl usually teaches five foundational skills, this year’s Summer Club only focuses on four: problem solving, critical thinking, confidence, and communication.

We left out compassion because this is an online program (thank you,COVID!), and we usually host offline field trips and workshops to work on building the Rookies’ compassion which we didn’t feel we could replicate well in an online learning environment. To measure these foundational skills, we used the Outcome Stars method.

For each foundational skill, we asked theRookies to rate (from 1-6) how much theyagreed with five different statements thatcharacterized that skill.After the program ended, we asked themto rate the same statements andcalculated the difference in their pre- andpost-program scores to measure theimpact we had on the Rookies on eachskill. For more information aboutOutcome Stars, check out their website!

Impact is really tough to measure, as it’s such an abstract concept (and this was our firsttime attempting to dissect it). We researched proven methods industry leaders used toanalyze impact and modeled our impact framework off those. We decided to use pre-posttesting to analyze the quantifiable outputs and outcomes and hope to do another surveyafter 12 months to measure long-term impact.

Number of participants who go through the program
62%

A 62% increase in the total number of Summer Club registrations as compared to the previous Summer Club.

Average delta of pre/post competency test in Communication skills

A 3.33% increase in average scores of communication skills based on the pre-club and post-club competency test.

Average delta of pre/post competency test in CriticalThinking skills

A 62% increase in the total number of Summer Club registrations as compared to the previous Summer Club.

Average delta of pre/post competency test in Confidence
3.37%
A 3.37% increase in average scores of confidence level based on the pre-club and post-club competency test.
Average delta of pre/post competency test in Problem Solving skills
A 3.57% increase in average scores of problem solving skills based on the pre-club and post-club competency test.

Here's what we found

Objective 2

Create support system and safe place for young women to learn

When we envisioned whatSummer Club would look like, we wanted it to be, first and foremost, a supportive environment where young girls could explore new opportunities without judgement.

We wanted to introduce our Rookies to as many inspirational role models as possible, and we created an ecosystem where everyone in contact with the Rookies throughout the program Mentors, TAs, speakers, and Volunteers fit that description.

We also used our retention rate as an indicator for this objective. We figured if Rookies were happy with the program and came back for future programs, it meant they felt comfortable to learn with us.

(-1%)

93%

Satisfaction score of the program decreased to 93% from the prior Summer Club, which was 94%.

31%

of rookies this Summer Club arereturning participants from otherGeneration Girl programs.

62%

increase in the number of role models introduced to the Rookies, as compared to the prior Summer Club.

Here's what we found

Objective 3

Strengthen Generation Girl’s relationship with industry leaders to increase brand awareness and network

The Generation Girl community was founded by young professionals in Indonesia’s tech ecosystem, so we pride ourselves in being well connected with industry leaders. Our partners and sponsors are always looking for the next generation of tech talent, so Summer Club is a great way for us to connect them together.

8

A total of 8 speaker sessions were held, remaining constant from the previous Summer Club.

+8%

increase in the number of partners who 8% give post-program resources or merchandise to the Rookies, as compared to the previous Summer Club.

Total impressions of Generation Girl’s annual Summer Club in social media:

From June 2020 (Summer Club 2020):
1773
Twitter: 1362
From June 2021 (Summer Club 2021):
3266
Twitter: 446

Here's what we found

Additional Data

We decided to gather a few more data points for the report. Although these didn’t directly align with our top objectives, we found some interesting insights we thought we’d share!
Increased knowledge of topic

+37%

From the survey completed by the Rookies, we found that there was a 37% increase in Rookies' knowledge of the topics they had taken.

Percentage difference from pre to post-test of likelihood to go into STEM

Pre-Summer Club, 43% of Rookies were interested in pursuing a career in STEM. Post-Summer Club, the number rose to around 53%, which shows a 10% increase in the number of Rookies who are interested intaking a career in STEM!

IPA (Natural Sciences) vs IPS (SocialSciences) vs Other of Rookies

Summer Club 2021 is dominated by Rookies whose majors (or soon-to-be majors!) are in the Natural Sciences, which is about 82% of the participants.

Percentage of those inclined to continue learning post-program

88%

88% of Rookies who have finished their class, are inclined to continue their learning on the topics that have been taken previously.

Where do we go from here?

Reporting is great, but we want to create actionables to improve our programs andimpact from the insights of this evaluation. We chose three pain points to focus on:
1. How to improve future Rookies’ Summer Club experience
2020

😀 94%

2021

🙂 94%

This year’s SummerClub satisfaction score was slightly (like, very slightly) lower than last year’s.

Very few Rookies use the post-program resources that we, along with our partners, provide. Perhaps one of the factors why Rookies are not taking advantage of our partners’ resources is because we simply give it to them without giving context on how they can be used. So how can we ensure that more of these products that our sponsors and partners have donated will be used by our Rookies? In future programs, we hope to integrate these resources into the curriculum to show the Rookies how these resources can be useful and, hopefully, increase the Rookies’ willingness to continue using them post-program.

When we dug deeper, we found that the Rookies were least satisfied with the speaker sessions, with Rookies giving the sessions a score of 89%.

To improve this score, we will give Rookies more autonomy and get them involved in the planning process for speaker sessions through social media.

This way, the topics and speaker ideas are tailored to their preferences as they come directly from the audience themselves. We also want to experiment on other types of webinar sessions, as we have been using the same question and answer format for all of our current speaker sessions. We’re hoping by doing this, we will keep these workshops engaging and reduce Rookies’ webinar fatigue!

2. Generation Girl’s roadmap for future programs

It was so inspiring for us to see how a short program like Summer Club, which only lasts between three to six weeks, can impact our Rookies.

In such a short amount of time, we were able to improve their foundational skills and even increase their interest in wanting to pursue STEM as a future career! If they respond this well to such a short program, imagine how we can impact them with even longer programs!

We have developed and are currently piloting a semester-long (five-month)program for high school students to build on the introductory classes we teach in Summer Club. Stay tuned for more information 😉

3. Generation Girl’s roadmap for future programs

For a first attempt, we think we did a pretty great job on our very first impact evaluation! However, we know it’s not yet perfect, and there’s always room for improvement. As mentioned above, our biggest limitation was the lack of data we had to create baselines for each indicator. As we continue to become a more data-driven organization and regularly use this framework with other programs, this problem will hopefully resolve itself.

Our method of pre-post testing is effective in understanding the short-term impact of the programs, but there must be a longer-term evaluation of the Rookies in order to see how this program impacts their future. Was Summer Club a deciding factor on the career path they choose? Did the foundational skills we introduced and practiced in the program help them in other aspects of their lives? We hope to conduct more surveys in the future, but for now, we can only say that the impact analyzed and reported here is short-term.

Thank you

contactus@generationgirl.org | www.generationgirl.org | @generationgirl.id
Click here to download the Summer Club 2021 Impact Report PDF with appendix