From the Trenches: Tips and lessons from herding data cats
Practical tips on running a Data Community.
From the Trenches: Tips and lessons from herding data cats
Read time: 9 minutes.
I recently saw this quote on LinkedIn - “I’m always surprised how many big orgs don’t run data communities of practice”.
And even if they do, are the facilitators and participants enjoying it, or is it just a tick-box exercise for some?
Well, I do run one and I enjoy doing it. And from anecdotes I hear, participants seem to enjoy them too!
Thus, I’ll share some practical tips on how to do it.
These community sessions could be called differently depending on your organisation: Guilds, Communities of Practice (CoP), Centres of Excellence (CoE). Ours are branded as “Data Engineering Chapter Time”. Don’t be fooled by the name though, we commonly cover other areas of the broader Data Ecosystem as well (Analysis, Machine Learning, Governance, and more).
In this article, I will share some practical tips that helped me achieve that.
Why the cat herding analogy? I thought this metaphor is apt given how independent-minded, unpredictable, smart and mostly introverted data professionals generally are.
How did it start
(you’re welcome to skip to the next section for practical tips)
When I joined the company in 2021, these sessions were already in place (though I’m still not sure when they began - one for me to find out). Attendance was even set as one of my performance goals. With MS Teams meetings and no cameras on, I quickly realised that wasn’t something I could game, surely.
I did enjoy attending, especially when led by our Data Warehouse expert - a veteran of nearly three decades - and I soaked up every word. But often, sessions covered topics my team had no exposure to, like “Platform XYZ upgrades,” leaving me wondering: what is this platform, what does it do, and why should I care? If I were lost while genuinely trying to engage, how did others feel? Other times, there was no agenda, and people were randomly called on to share “what’s new,” which usually led to several seconds of awkward silence or “brb” messages in the chat. It felt like untapped potential.
A few months later, when the expert asked for help running these sessions, I volunteered immediately. He agreed, and I was excited to step in. His time and expertise were in high demand elsewhere, so from that point, I had the privilege of building on his strong foundation - with a focus on boosting engagement and making the content relevant for the whole community.
Start with “Why”
Start with “Why” - cliché, yes, but still relevant (thanks, Simon Sinek). Why what?
Why run these sessions? One hour a week from a Data Engineer is a significant investment, so the purpose needs to be clear. My starting point was simple: “As a Data Engineer, I want to learn anything that might be important to my role.”
Why would anyone want to attend? It could be to tick off a performance goal - or to learn something new and feel part of the community. I left the choice up to each person, with one key change: send topics in advance so people can decide if it’s worth their time.
Straight away, patterns emerged - which topics attracted attendees, and which groups of engineers were tied to different parts of our tech stack.
Mixing in product thinking
I have carried on with the above adjustment for some time, and feedback was positive.
However - I did occasionally run into someone in the company who worked with data in some way, and was surprised how little they knew about things like: our broader tech stack, what is available, how to find it, who to contact and so on (“oh, is that where we store our JIRA data?”).
A bulb went on:
Facilitating these sessions is a valuable service.
Am I reaching all the potential customers? In other words, does it need to be limited to just Data Engineers?
I don’t track or measure reach and engagement.
Focus on those who want to be there, not those attending to meet a performance goal.
This realisation got me to re-think some key aspects of how the forum is run.
Here are the tips that proved most effective.
Practical tips to boost the forum engagement
Tip #1 - Review and update the list of invitees
Who are our current and potential future customers?
What you’ll need:
List of invitees (email addresses) from your Outlook / Teams invite (just open and copy & paste in).
Staff table of your company (if no direct access, ask for one-off extract) from HR system / Active Directory with: Name, Email address, Job Title, Team / Org Unit name (nice to have)
A tool to mash the data up (I used a mix of PowerQuery in Excel and SQL - depends on what you have available in your company)
What to look for:
Look at the team of existing attendees - are there others we could reach out to?
Look at the job titles - teams often have multiple titles within them
(Data or Platform Engineers, Scrum Masters, Product Owners, Architects) - why would I exclude the others if they work for the same team?Look for anyone with “data” or “analytics” (etc) in the text of their job title - where are they in the business?
Any other tech individuals who deal with Data or Data Platforms
(DBAs, Solution Architects etc.)
This exercise grew the invitee list from 60 to about 300. It sounds daunting, but with a clear agenda and transparency on expectations, it works.
A publishable participant list, including the reason they’re invited.
My categories are:
Member of the Analyst Team (key Customers to Data Engineering)
Data SMEs from Other Business Units
Project / Programme Managers of key data initiatives
Chapter Leads from other Chapters (cross-pollination)
Member of the Data Platform team but not a Data Engineer
Architects with a focus on Data solutions
Others with "Data" in their title
Risk Consultants with a focus on "Data"
Infrastructure Engineers
Others with "Analytics" in their title
Tip #2 - Be clear on the purpose.
You might think this should have been Tip #1 - and you’re probably right. For me, the real purpose only clicked after reviewing the invite list.
The biggest opportunity was breaking through organisational silos, each packed with professionals who share an interest in data. No surprise there - Data Engineering cuts across almost every domain, tech stack, and team in the business.
The solution seemed obvious: create a shared forum to connect everyone with this common interest.
Our meeting invite now reads: “Facilitate cross-business-unit communication and learning across all teams dealing with data and analytics. We strive to break down organisational silos. Silos are for grain, not for people.”
Tip #3 - Be clear on expectations, rules and etiquette
Attendance
One major change I made was to stop requiring attendance. Our meeting invite says: "Since we can’t mandate your attention, we’re not mandating your attendance." I’d rather have 10 engaged participants than 100 tuned out ones. That said, even if someone just wants to drop in for a few minutes, that’s fine.
Rules and Etiquette
We don’t have formal rules - just a shared expectation of respectful conversation. If the agenda is tight, I’ll park questions for later and make sure they’re revisited.
Recording Policy
After surveying the group, I settled on recording sessions but switching it off for Q&A. Occasionally, we run open discussions with no recording at all to keep things candid.
Schedule
We meet every Tuesday from 2-3pm - avoiding mornings (engineers’ most productive time) and the end of the week (when energy is low).
Tip #4 - Keep it interesting
In short:
Pick relevant and timely topics
Change things up - speakers, themes, formats
Relevant and timely topics
Easier said than done. My approach is to pursue what interests me and bring others along. There’s no shortage of subjects to explore in the data world, both in the industry and within a large enterprise.
Themes we cover include:
Technical concepts (schema evolution, naming standards, etc.)
Cross-team sessions with guest speakers from Software, Platform Engineering, Business Analysis, Risk Management
Stakeholders from other business units and their data needs
Solution presentations and feedback (built something cool? Show it and inspire others)
Architectural and governance deep dives (Data Mesh, Medallion, Data Contracts)
Industry trends
Significant project or technology roadmap updates
Change things up
I’m lucky to have brilliant co-presenters, each bringing their speciality and spark. Rotating topics helps, and the forum is open to anyone who wants to present their team’s work, solution, or challenge. These sessions often spark the real “aha” moments.
Tip #5 - Lean on the community
As mentioned in the previous tip, we keep a solid mix of presentations from within the community - engineers, analysts, and anyone who has built something useful.
For feedback, we run an annual survey to find out what’s working and what’s not. Key changes to the forum, such as time slot or recording policy, are discussed openly, and feedback is invited.
We also keep a topic backlog in SharePoint where anyone can add or vote on suggestions.
Tip #6 - Measure attendance
While attendance shouldn’t be the sole measure of success, it’s still a useful metric to track.
I use Microsoft Teams attendance reports - downloading CSVs to a SharePoint folder each week (takes about 10 seconds manually, though automation is possible). A simple Power BI report then combines the data to show both high-level stats and detailed views of which participants prefer which topics.
If you have access to staff hierarchy tables, combine them to see attendance by team or business unit. I focus on the ratio of “core chapter” vs. “others” for each session, alongside total numbers.
The report also includes a list of past sessions (topic, link to notes) and confirmed or planned future topics, tracked in Excel. Combined with attendance data, this gives a clear view of engagement per topic or presenter.
Finally, I embedded the report into the MS Teams chat so the upcoming agenda is always accessible and easy to find.
I will leave a detailed description of the report template for my future articles.
Lessons learned
After putting these changes into action, here’s what I’ve seen and learned to date:
Average attendance increased from 30-50 per session to 80-150.
Attendance fluctuates with seasonal factors like school holidays or major events (upgrades, go-lives) - and that’s fine.
As numbers grow, fewer people speak on mic - no surprise, most tech people are introverts. Chat helps, and I monitor it closely.
Big announcements (migrations, roadmaps, new tech readiness) draw the largest crowds - the community cares.
Weekly sessions take commitment, but engagement and community-led presentations make it sustainable.
The forum is a great space for engineers to practise presentation skills.
If you like technical work, being called “the person who talks all the time” can feel odd - but it’s no different than calling a developer “the person who presses buttons all day.” Ignore it.
Impostor syndrome can creep in - if it does, lean on colleagues or trusted resources.
For niche topics, special interest groups (separate dedicated sessions) work well - we currently have one on Scala for example.
The road ahead
My goal is to keep improving both the format and content of these sessions.
Key focus areas:
Revisit the invitee list regularly to find untapped potential in the organisation, not just as a one-off.
With the current “AI” hype cycle, expect more practical AI sessions tailored to Data Engineering.
Bring in more external guests - the 2024 session with Shane Gibson on the Information Product Canvas was a hit.
Plan and deliver an internal hackathon - it’s a frequent request and now in the ideation stage.
Thanks, see you next time!
Credits:
Illustrations taken and adapted from https://storyset.com