The St.

Paul City Council Audit Committee to order.

Roll call, please.

Um Committee member Dilworth.

Virtual uh committee logic.

But let's let's let her say here.

Oh, I think we can't hear you, Stephanie.

Um she's not muted.

Oh, you're not sound on high enough from the screen.

Maybe call everybody else first.

Okay.

Um committee member Foster.

I'm here.

Committee member McBay.

Present.

Chinaker.

Here.

And um Councilmember Coleman?

Here.

Perfect.

So we're just waiting on still can't hear you, but we're seeing what's wrong.

You can hear us, I take it.

Looks like yes.

Okay.

Hopefully it's on our end.

Um give it a second and then we'll need to face the mic towards you.

Also, it picks up your voice.

Oh, sorry.

Okay.

They're working on establishing audio for Stephanie.

Okay, okay.

So we don't have it fixed yet, but we'll just acknowledge that.

But Ms.

Delworth is here and can hear us.

Okay, great.

All right.

Uh well, welcome everyone to our November audit committee meeting.

A lot has been going on, and we have a lot on our agenda, so we're gonna dive right in.

Um first we're gonna welcome up Dr.

Heather Bret to give us an update on the data practices study scope of work, which is our not current audit, but our about to be current audit.

Um, Ms.

Vang, did you want to add anything before?

No, I just want to say that this is our third study of Wilder Research head by Dr.

Brett.

Welcome up.

All right.

Uh let's see.

Thank you.

Uh thanks for having me back again.

Um, Nia and I and uh your legal team and other folks from the city have spent a lot of time together um since the last time I was with you.

Um and we have done some tuning and some modification of the scope of work.

Um you all have given some additional uh input, and so this is really just an opportunity for me to sort of say, here is the revised scope for us to get kind of a final stamp of approval so that we can really um get earnestly after it.

So that's that's our plan for today.

So all right, here we go.

Let's see if we can go to sleep.

No, yeah, there we go.

I got it.

Uh so we're just gonna do basically the scope update, um, sort of that refresh.

Um so again, just kind of a reminder of our purpose, and we have tuned our purpose, and again, this is really in response to some of the feedback that we got from folks across the city, and that includes folks from the mayor's office.

It includes a variety of members of the legal team.

So um to assess the effectiveness, efficiency, and compliance of the city of St.

Paul's data practices request processes, tools, policies, and user experience with a focus on the city's use of GovQA, which is really the sort of management system for those data requests and its alignment with the Minnesota Data Practices Act requirements.

Um so that's kind of the sort of the refreshed version of this purpose, um, which feels um, I think comfortable enough for everybody at the table so as to give some input information to the audit committee so as to give input and information to the city, um, but so as not to um uh present any additional risk.

So I'm gonna do what I have done uh sort of previously.

I'm gonna talk about some key research sort of questions or key research areas.

I'll talk about sort of the key stakeholders, I'll talk about kind of that data collection approach, and then I'm gonna talk a little bit about our adjusted timeline, given it's November, and I think I came 252 years ago the last time around to talk about this.

I'm kidding.

So I came in the past and we've done some modifications.

So in the first sort of area, so you'll see A1, I'm really just talking about kind of area one.

Um so we really said this talks about GovQA.

Again, that's the tool that the city is using to do all of this kind of tracking and movement around of what those requests look like, the processing, the tracking and the movement and the closure, the communication with end users.

Um so talking about its capabilities and its internal use.

Um, so key questions here.

What is that standard capability set look like?

Um, how has the city of St.

Paul configured or customized what GovQA looks like for its purposes?

Um, how is that use documented for city staff?

So, what are those procedures, training materials, process maps look like?

Um, and does that GovQA system provide sufficient management controls to really efficiently and effectively process data practices requests?

So a little bit of uh kind of you know, is this tool sufficient for the needs that the city has?

So that's sort of the kind of tool capabilities portion of how we're thinking about this work.

The second part of this reporting and data trends.

So if you'll remember, this was one of the things that you all were really curious about.

Um so one of the open questions, what kind of reports are available to the city staff through GovQA?

Um, which reports are vendor provided.

So some of those reports were sort of out of the box reports that just showed, um, sort of up for the city staff, which were created specifically for the city.

Um, and then what do aggregate GovQA reports reveal about request volumes, trends, timeliness, and patterns since GovQA's implementation.

So, and credit to Nia who has already started to dig in, have conversations with the vendor about actually getting all of that stuff sort of pulled and spun up for us.

Um, and again, I will say we are looking at aggregate.

Um, we are really not interested in for all kinds of reasons looking at any line level information here for the purposes of this.

Um, so we moved into kind of this, um, and I think one of the pieces of feedback that you all have received since the last time we were here, and one of the pieces of feedback we have received directly from some folks in the city is about just some of that resident-facing experience, and so wanting to focus really that resident-facing experience on the portal as a way to kind of narrow what that scope could look like.

Um so thinking about the portal that residents use actually to sort of request data from the city.

Um, so what online portal this do city residents use to submit data practices requests, how does that portal function?

How do residents receive correspondence and data from the city?

What is the experience of residents or end users when navigating the online portal?

And then how to frequent kind of low need high use requesters.

So we have a particular set of users that we have been pointed to perceive the city systems compared to other jurisdictions that they routinely interact with.

Um, really just getting a sense for how is this user experience in comparison to their user experience with other jurisdictions.

Sorry, Ms.

Coleman.

Thank you.

Just a quick question on that.

Do we anticipate that our low-need high use requesters are making frequent requests in other jurisdictions?

Is there may be some.

Okay.

Yeah.

Thank you then.

And I don't know if we will go as far as to name the individual, but we we do know of a few names that do that.

Great.

Yeah, thank you.

Just a curiosity question.

And then policies, training, and compliance.

And this is one where I would say we are trying to be particularly pointed.

So what training programs and guidance are provided to city staff responsible for processing data practices requests.

Are these training programs and guidance adequate to ensure compliance with the Minnesota Government Data Practices Act?

How do city training programs and guidance compare with model forms, procedures, and guidance issued by the Minnesota Department of Administration's data practices office?

So again, we're focusing really on that kind of training piece, that guidance piece, that part here in this sort of particular kind of section, section four.

And then this sort of final area is really that sort of comparison with other jurisdictions, which is something you all were really interested in.

And this is a place where the mayor's office and your legal teams had a whole lot of comfort and sort of a broader sort of sense for latitude here.

So doing some work finding out what are those processes, policies, procedures look like in other Minnesota Minnesota jurisdictions.

And so we're gonna take a look at Hennepin, Ramsey Counties, Minneapolis, Bloomington, Duluth.

What kind of request management tools and online portals do they use?

So just to get a sense for what does it feel like to do that request online, and then what kind of tools are they using internally for doing kind of that traffic control management?

Um what publicly available data exist regarding their request handling.

So we're not going to go so far as to say hand me over your internal data, but we are gonna look and see what sort of data are publicly available.

So log statistics, performance measures, anything like that that might be available that might be a metal a bit of a model for St.

Paul to be able to take a look at.

Um then uh as we alluded to before, we're gonna take a look at kind of that peer-reviewed literature and then what we call gray literature, so just kind of best practices literature, what does it look like for other jurisdictions across the country exist for you know, kind of the municipal public records programs nationwide?

Um so taking a look kind of what else, what else is out there and what might be good additional guidance frameworks, um support for the city to consider as you're as you're moving forward.

So those are kind of our big um areas of focus.

Again, those have tuned a fair bit since the last time I was here for all the reasons that I I suspect you understand.

So we do have a much wider and deeper set of specific names attached to this, but I'm gonna talk at kind of a high level about the stakeholders.

So obviously the city clerk's office, you all did a fabulous job celebrating her.

So I just want to say that for the record.

It was fun to see, fun to see that in the paper.

Yeah, yeah, yeah.

Um council committee staff, obviously, um, we'll be sort of doing that door knocking.

Um, and then uh frequent requesters to the Department of Safety and Inspections through the city online portal.

So we'll be sort of requesting, and we have in hand the names of some of those humans.

Um, and then sort of state and external entities, obviously the data practices office, um, the state auditor, so taking a look at sort of what are those regulations, what are the what is that guidance look like, League of Minnesota Cities, um, subject matter resource expert?

Um, and then as we alluded to those other Minnesota jurisdictions, so wanting to kind of understand what are those communities or counties engage in from a sort of uh sort of best practice or sort of what are their what are their plans, policies, procedures, guidance, um, online portals and sort of data tracking systems that they use internally.

And Dr.

Burt, before you keep going, I know this was something that we I got a lot of feedback on in terms of who should be consulted as a stakeholder, and I see in this uh handout that we have that you have sort of written under key stakeholders that the audit committee has agreed members of the public should be included as key stakeholders and that consultation with the data practices office and Minnesotans for open government will strengthen the scope and methodology.

I see the data practices office written here as a specific entity.

I don't see Minnesotans for open government, and I'm not sure I understand is frequent requesters to Department of Safety and Inspections through the City Online Portal the same as members of the public who have made data practices period.

They are.

Yeah, yeah.

So I think we're thinking about that as kind of the subset of members of the public that we're gonna sort of focus on, kind of those frequent requesters as folks that we're gonna want to get a sense from.

Tell me about that experience of using the online portal, tell me about that experience of using the online portal.

Um if you've used other online portals, how does the city of St.

Paul online portal sort of compare?

Tell me about the sort of responsiveness, timeliness, um, those kinds of things.

Um, so that I think is kind of that public feedback.

Um, and we did decide we're gonna go with a survey there as opposed to sort of a more open-ended qualitative set of feedback for um intentional purposes.

Um could you state your first sort of question first?

Just so Department of Safety and Inspections is that's regardless of what you're asking for data about, that's where you go, Ms.

Vang.

Yeah, I just want to make sure we're not limiting it to.

When we pulled the names of people who commonly asked, or what I call franquet requester, found out that a lot of the requests was routed through the department inspection.

So they were the frequent folks that we identified.

So perhaps we could also look at all the other department, but these are the one that had the highest volume that came in.

Okay.

Yeah, I mean, I think I I guess as long as we're doing a survey, I'm not sure why we necessarily only need to talk survey those who frequently requested.

It might be interesting to cast a broader we could broader canvas.

And I and I don't want to limit it, I don't see why we would limit it to asking of one department.

You know, it seems like we're trying to understand our overall data project.

Yeah, it wasn't intentional.

Um Chair Naker, is just that when we pulled up the names, is it just happened to focus on that department?

So we will try to expand that if that's the case.

Great.

And then I just was pointing out that I don't see Minnesotans for open government in the list.

I'm just noting like there's a paragraph and then there's a list, and I want to make sure I'm understanding.

Yep, yeah, yeah.

So we can do the alignment of the state and external entities with that paragraph.

Yeah, yeah, yeah.

Okay, thank you.

We can keep going.

Yeah, and I think I left the paragraph in from the modified scope that came from you all.

So yeah, I can do some some syncing and some tuning.

So I assume that was language straight from you all.

So it's like, oh, I'm gonna leave that in.

So yeah.

And yes, we can uh, you know, we're gonna do an electronic survey platform so we can talk about how wide we want to do that and the period of time that we want to think about how long ago those requesters sort of used that online portal.

So I don't know that we want to go back as long as GovQA has been in existence, mostly because we actually want folks to have some recency about that experience.

So let's connect and think about how you know how wide we want that from a from a survey window vantage point.

So, but we certainly could go with a wider capture of humans.

Okay.

Let's see.

Um yeah, so we've got sort of a variety of ways that we're thinking about sort of the review.

Um, so that kind of document and system review.

Um, and so you can see kind of the you know, review of everything that sort of the city's got.

Um Nia's been starting to assemble all the kind of training materials and all the kind of standard out-of-the-box stuff that is coming from GovQA, um, taking a look at all that sort of performance data, um, and then kind of taking a look at kind of what are the model policies, sort of the record retention information, kind of the standard things that we want to be taking a look at, um, just so you all have in hand, kind of here's the standard set of guidance that would be the standard set that you might be thinking about and sort of considering, um, especially as the city is sort of assembling that new team to really be to really sort of think about um addressing your uh um sort of data practices and how you're making um sort of um folks across the city feel like you are as responsive as possible in a way that feels as supportive as possible, given you know what is really an increasingly wide and deep volume of requests coming through to the city.

So uh, you know, taking a look at that literature, taking a look at some of the sort of uh benchmarking.

So again, um, like we said before, kind of we'll do we'll have our library services team kind of take a look at this.

Um, you know, and take a look at, and we've already started doing some of this um sort of review of what those publicly available sites look like for municipalities across um Minnesota, uh, and then taking a look at, you know, um, and again, appreciate you all sort of referencing and sort of sending this our way.

So again, um like we said before kind of we'll do we'll have our library services team kind of take a look at this you know and take a look at and we've already started doing some of this sort of review of what those publicly available sites look like for municipalities across um Minnesota and then taking a look at you know um and again appreciate you all sort of referencing and sort of sending this our way kind of there is already some public precedent um for the kinds of standards that that the city might be thinking about um and so making sure we're just tapping into what those standards look like so the stakeholder engagement um and again we'll think about kind of doing uh broadening that survey just from a volume and an end vantage point um uh we've already had a couple of conversations with Nia um uh and you know former city clerk um before she departed um so thinking about some additional conversations just to just to make sure we have a really good handle on exactly what that process looks like um uh and then uh thinking about additional interviews with some of the staff who are responsible for um kind of that data request process so Nia's been great at identifying who from each one of those departments is responsible for that work um and again we are going to keep the scope of those conversations inside the scope of what this um of what this particular audit is focused on um we will uh do interviews with folks from each of those Minnesota jurisdictions um just to understand what their processes look like um and then again have that consultation with uh League of Minnesota cities and the department administration that particular office so again just to understand best practices guidance um so really just to get a handle on um kind of that you know in in order to really sort of develop for you all kind of here's the book of best practice and guidance that could be the reference point as that new team is sort of put into sort of into deployment uh timeline we have just kind of shifted it forward um a little bit so kind of we're in that scope finalization we're gonna step into that um and we've already started some of that like document and data review starting to do kind of those interviews develop that survey we're gonna kick off that literature search um we'll keep digging into analysis um really kind of wrap up we hope those interviews make sure that survey is kind of wrapped up in the field we're doing that synthesis of literature have that draft available in February and then you know be able to present that final report in March and again we'll do the same thing that we did before you'll have a chance to take a look at the report give feedback we'll present you'll offer additional feedback and then we'll do kind of a finalization of of that report um for for you all and then I think again once that team is on board comes on board has already been on board happy to do additional support with that team depending upon what is needed given their their feedback and and needs that they may have great thank you so much Dr.

Brett I want to um open it up for some brief questions we do have a ton on the agenda so want to see if colleagues have questions comments concerns but this is going to be whatever this is our last opportunity for feedback before the audit actually begins so Mr McVay.

Thank you madam chair um Dr.

Britt with regards to data collection approach um number three with stakeholder engagement I just wanted to make sure that our consultation does include the state auditor's office as well because when they are auditing our financial statements every year for the AC for and the CIFA for federal programs that sometimes they are actually going to be looking at our compliance with the Data Practices act sometimes with certain federally funded programs in particular and it might be helpful just to reach out with OSA while you're doing your field work potentially I just want to make sure that they're including that consultation.

I know they're one of the state uh state entities that's listed and I just wanted to bring that up okay yeah yeah yeah I think we um again I'm gonna put edges around that so happy to seek their guidance edges right consultation that way um what we won't step into is any of the like feedback or any of those kinds of things right so we're really looking for kind of the expert um here's the kind of general consultation and guidance that we offer to all cities so yeah but yes happy to add them okay yeah thank you yeah yeah yeah yeah great suggestin thank you Mr.

Austiner yes uh Chair Nakre um just a question on the survey with our external requesters I do agree with the approach that it should be broad and hopefully we get infrequent and frequent requesters um I did wonder what the frequent week requesters do we think we understand why they're requesting frequently or is that something that the survey is hoping to you know bring to light uh it's certainly something that we can include I suspect that some of the staff have a sense for some of that I wonder if we already know and I I personally don't know I would maybe like to know.

I wouldn't maybe like to know.

Yeah.

Yeah.

I mean, I do think it might be helpful for, and we talked about we're gonna keep that survey entirely closed-ended.

Yeah.

And so I do think it would be um something where it would be helpful for them to give us in there in those sort of closed-ended responses, their reasons for um, you know, for those requests.

So maybe how many requests have you made in the last year or something?

You know, yeah.

So what kinds of requests, number of requests, right?

You know, just kind of that.

I I do think kind of, you know, perception of that portal usability of that portal, right?

You know, kind of the the the kinds of basic sort of user experience questions that I think we want to get at.

But um, yeah.

But I do think from their lens, um, because we should be able to pull some type information from GovQA.

Again, I'm conditioning that because the back end of that system, as Nia has told me, and I am discovering is a little trickier than I think what I was hoping.

So, but she has been living this.

So when you look at the aggregate, are you gonna be able to see how many requesters and um how many requesters or you know, for each requester, how many requests they made, you know, you can de-identify it, but just say how many frequent requesters do we have, how many one-time requesters kind of a weird answer.

Um, if they create an account, then you can you have to search if you have a known person.

You can search to see the type of request they made and how often they requested.

Um, if you don't have a named, because you can be an anonymous.

You can do an anonymous, yeah.

And we get a lot of those too, but you can't tell.

Yeah.

So learned something new.

I can already tell we're all gonna learn a lot about this process just.

Yeah, right.

That's we're saying the scope of the audit, not even auditing.

Yes.

Um did you have more, Mr.

Last?

Yeah, that's very informative informative.

Thank you.

Yeah.

All right.

Any other discussion?

Otherwise.

Just one thing, yeah.

Thank you, Madam Chair.

I just want to say thank you, Dr.

Britt.

I'm really excited about this audit, and this revised scope is just wonderful.

So I'm nerding out over this already.

Good.

Good, good, good.

I want to say um thank you for your patience.

Uh thank you for inviting half the city to get together to help advise us on how to negotiate getting to a place that feels I think comfortable for everybody around the table.

So that was um, that was not an easy um set of conversations, and so I really do appreciate your patience and your support and your support in gathering all of those humans together.

So I think you're at a good spot.

I think it will really especially be supportive as that new team is sort of stood up and supportive for the staff and thinking about huh, what kinds of opportunities are ahead of you.

So thank you so much.

We look forward to the steps ahead.

Um, and by the way, Ms.

Dilworth online, if you raise your virtual or real hand, I'll know that you I just want to make sure you know you're always able to indicate that you want to talk and then keep an eye on you.

Awesome.

Thanks, Dr.

Britt.

Thank you.

Um, so as we move on to the next topic on our agenda, kind of we've just launched our current audit, and as you know, if you look at our audit committee handy dandy complicated yet helpful schedule.

Um that that is underway, we kind of move into opening up and looking for our next topics.

Um, and as we talked about last time, what we're hoping to do this year is have sort of one big idea gathering phase, which is now through December, um, and then choose our two topics for next year, so that we're not running a twice a year topic engagement or topic farming harvesting process.

We're doing that once, choosing two and then going through the year based on those two.

Um so that's why we're moving to community engagement now, because this is the phase in which we are soliciting those topic ideas for the harvest.

And I want to have say a huge thank you to Mr.

McVay, who is ably stepped in where Mr.

Farnsworth has stepped out um to help us with the community engagement process.

We talked about um some ideas last time, which um I think Tony and Nia very beautifully summarized here when we were discussing how we should get the word out about what we're doing and about the topics that we're trying to gather.

Um but I want to turn things over now to Mr.

McVay to share the work that he's been doing and lead this part of the discussion.

Thank you, madam chair.

Uh I will begin by saying I reached out to the district councils that did not respond to our prior correspondence.

I mean, I should back up a bit.

The council, the committee itself reached out to all 17 district councils back in November of 2023, is my understanding.

Long before my time.

Uh and many of them did respond and participate in the uh uh the topic selection survey for residents.

The um there hasn't been that kind of engagement with the district council since then.

Um six of those district councils never actually um return any correspondence with um our committee.

I took the liberty of reaching out to those three um district council or those six district councils, three of them have responded to me at this point, either um over the phone or through um email correspondence.

They are the West Seventh Fort Road Federation, which is District 9, um, Como Park District, Como Community Council, District 10, and uh District 15, the Highland District Council, and I appreciate those folks that uh were kind enough to return my correspondence.

I have meetings with those three district councils um next month, actually with Como on December the 17th, and then again with Fort Road West 7th on January the 12th, and then again January the 8th with the Highland District Council.

Unfortunately, I've not gotten any other communications from the other outstanding three district councils.

When I found out that we hadn't been in contact with any of the other district councils since November of 2023, I decided that it might be a good idea to also get in touch, follow back up with the the other 11 district councils, and the response rate has been uh to put it gently uh lackluster.

Uh I did receive a phone call from the executive director for the Union Park District Council, and I'm we've been playing a little bit back and forth.

I'm hoping to get in touch with her sometime this week yet to find time to meet with her board, potentially yet um next month, if not in January.

Uh otherwise um the district councils have not been very responsive, and it might be partly because I'm sort of a nobody.

I as a community advisor, and it would be very helpful.

I am an I am a nobody.

I I am a member of this audit committee, yes, but I don't have say the weight that the council member does.

And I was wondering if it might be a good idea if to speak one-on-one with the council members and see if any of them be willing to reach out to the and their staff to the some of the district councils are being a little non-responsive and see if we can kind of get some communication going.

Obviously, I know all the council members are very busy, and I'm here to try to lighten the workload a little bit for you.

So uh I'm more than happy to still be going to all these um district councils and meeting with them and letting informing them what our work is and trying to get the public to attend these um town halls during regularly scheduled board meetings and uh see if uh we can get more participation participation, excuse me, with uh the um with just uh getting more audit topics out there because obviously if we have a larger data set to work with uh the better we can actually be at having more timely um decisions on what kind of audits we want to have and uh that we can recommend to the full city council, and so um I did reach out to Insight St.

Paul and to St.

Paul Strong, and both those organizations did actually complete the survey.

The steering committees for both those organizations were very interested in our work as an entity, and so um actually ecstatic was the response I would categorize it as.

So um it's nice to be loved.

It is nice to be loved.

Uh but uh other than that, that is the extent of my um outreach at this point in time.

I do hope to continue this.

This is gonna be an ongoing thing.

I imagine that I will be having meetings um late not this month at this point, but um December, January, and perhaps even into February.

And I know that doesn't exactly line up with our desired timeline, so maybe I need to fudge it a little bit, but I will keep working on it.

Uh thank you again for your leadership on this.

It is not easy work, but it's such important work.

So thank you.

Um I saw Ms.

Coleman in hand.

Thank you.

Thank you.

I will echo the thanks because it is it is not easy, but it is so appreciated.

Um, and I apologize because I got distracted comparing these dates to my calendar, so I am having some listening struggles.

Um is it helpful?

So just thinking about the district councils, I have five district councils in my ward, so super happy to make sure that this is on the radar of all of them.

Two of them are on this list.

Yes.

Is it helpful for like these meetings are that are on listed on here?

Are they just normal community council, district council meetings?

Yes, like should I show up at the Como Park District, like the D10 meeting tomorrow and talk to them about audit committee?

Is that helpful?

If you want to, um, here's the thing.

They already have those the meetings that you see there are scheduled at this point in time, they're on the calendar.

Those meetings are full.

They have full agendas for those evenings.

And so that is why I will not actually be presenting until um next month at the earliest with the with Como, for instance, with the Coleman Community Council, but with the other district councils, it'll be in January at the earliest, and that's why.

So you have if you are more than welcome to, ma'am, if you wish to do so.

I I don't know how appreciated it will be if their schedule is already fully packed, but just want to offer that if it is helpful, especially with the with D10 and D11.

I'm happy to take those meetings off your plate, be the one to go or you know, just coordinate or whatnot, and then um same goes with the other three district councils in Word 4.

Happy to connect with them, make sure that this is on their radar.

Um, and if it's also helpful to like loop me in on any conversation, happy to do that.

And then, you know, since we only have three council members on this committee, I know that leaves a lot of district councils uncovered, so maybe the council members amongst us can talk about how best to make sure that our colleagues are conveying this information to the district councils that they represent because I think it's you know, I'm happy to go over to the side, but I feel like they would prefer to hear from council members jobs at Yang, for example.

Um, Councilmember Coleman, if you would like to um come to any uh meetings that I will be attending that are in your ward, you are absolutely more than welcome to join me.

I would appreciate the company.

Awesome.

And Mr.

McIntyre, maybe um I appreciate that, Ms.

Coleman.

Maybe it would be helpful would be for you to sort of serve as air traffic controller.

So you let us know who you haven't heard back from.

And then to Miss Coleman's point, the council members can figure out, okay, how do we need to reach out to them in a in a different way, um perhaps to get their attention.

Um and then depending on if you're not, you know, I that 17 is a lot of district councils to go to.

I think you should also feel free to deputize us to to attend some of the meetings as well.

So I will most likely be doing that.

Yeah, yep.

I think you should you should do tell us what to do, but your owning the organization part of it is huge.

Yes.

Um great.

And then thank you for reaching out to Insight and Strong.

Um I'm looking at the other ideas that we had come up with last time.

Um I am wondering if so, Nia, can you Ms.

Vang, can you update us?

So the topic selection survey is that that's always live.

And yep.

And did we receive like a Facebook blurb or something for us all to help spread the word about it?

I believe I sent it to all the uh committee members and advisors on on this.

Um so if you didn't get a copy of it, I'm happy to reforward it to them.

I give you guys three options.

Um you don't have to use all three, but if you want to, um, by all means please help spread the word.

Okay.

Yeah, maybe resend that to all of us, and I think all of us should make an effort this month to put that out on our social media and our platform to platforms to let people know that we're soliciting topics.

Um what other ideas did folks have besides the district councils, besides some of these specific community groups?

Um, and besides all of us advertising the topic selection survey for that gathering in.

I know we talked about talking to the city attorney's office.

I think we're in a different environment now that we've had a mayoral transition, so I think staff is a little bit preoccupied and uncertain, rightly so.

Um so it may also make sense for us to hold on some of the conversations with staff.

Also may make sense for us to talk to our incoming mayor and her administration when they're set up.

We may we might need, despite our virtuous attempts to make this the calendar that always stays the calendar that we never be off from with a new mayor coming in.

We may want to take some time to find out from her what she would like to do and what she is interested in studying, because I think there may be things that she's very interested in learning more about at the city.

Um but what other ideas did folks have in terms of ways to get to solicit topics if they're not already captured.

And and specifically if there's any ideas you have that you yourself can help with, because I think it's all hands on deck.

Yes, Mr.

McKay.

Thank you, ma'am.

Uh so in Seattle, their city auditor's office, they actually have an ongoing what they call an audit fair.

And if I recall, they go around to um their um they actually participate in um like uh how do I put this?

And if I recall, they go around to um their um they actually participate in uh like uh how do I put this they go around to schools within Seattle and actually talk about the work that they're doing.

And the reason for that from my understanding is a lot of public schools in Seattle are used as polling places, and so um they decided that it'd be since a lot of folks in the um Seattle are already kind of affiliated with a given voting precinct, and they're kind of schools are central in their community in that sense that they go around to school districts or not school districts, but individual schools throughout the year and allow the public to come in and talk with them, and they have these um audit fare basically is what they call it, but it's to be all in all fairness is really more like a town hall.

Um that's something that could be done.

I don't know if we actually have the bandwidth to do that here with um our school district, perhaps.

The other um something else that I had that came to mind uh is um oh having a little bit of a that train just derailed a little bit.

Um maybe hopefully that comes back to me.

Uh I forgot what I was gonna say.

All right, I'm sure I'll come back to you.

Well, we're not going anywhere.

Yeah, exactly.

Um thank you for that suggestion.

And I couldn't see if I saw a hand or not.

Okay.

Um I love that idea.

It does seem like a high capacity sort of project, but um, but it's good to add to our list for such time.

And I'm almost wondering if there are, I don't know if there are newsletters or other ways of communicating with our the students, whether St.

Paul Public Schools.

Um I think it probably does require more of a presentation though, because otherwise you read something in a newsletter, you have no idea what the what the audit committee is.

And they do give a press uh presentation too, and it's very very broad.

It's for general public, so they try to gear it in a way that kids will enjoy it too.

Right.

So we need to see what that looks like if you can ever find the link to it, Mr.

McVeigh.

I'll dig into that.

I also actually have to um I before our committee meeting, I was talking with uh Ms.

Vang a little bit, and I need to still gather.

Um I was gonna share some literature regarding uh civic engagement and public audit with the committee, and I promised to do that months ago, and I still haven't gotten around to it, and that's that's on me, my little screw up there.

So I do apologize to colleagues here, but I will get to that at some point.

So I will make sure to include that as part of that um paperwork dump.

That sounds good.

You've been busy, so not to worry.

Um okay, so it sounds like we're talking to district councils, we're gonna send out our topic selection survey.

We have already reached out to some specific organizations.

Um we I also know talking to council members and their staff.

Um, and I think maybe uh council member Coleman and I can discuss how best to talk to our colleagues to get their ideas, um, especially post-budget process.

That's often when people are very much thinking about like these are things that I really want to dig into next year.

Um all with the goal of coming back to our December 16th meeting with a whole bunch of topics, which we would discuss and try to narrow to four for risk analysis.

That's our goal.

And I think Mr.

McVeigh, if you're continuing to engage district councils beyond that, that is perfectly fine.

Okay.

Um, this is ongoing work, and the more that they know about us the better.

Um so anything else to talk about with regard.

Did we want to look at the did you want us to review the presentation?

I wasn't sure.

We can absolutely if you want.

Um Mia or Ms.

Vang, I just want to say thank you so much for sharing this with me.

And for she um Ms.

Vang converted the original PDF form that um former community advisor Hornsworth had and brought it back into a PowerPoint format that could be then used as a deck slide that can be edited.

So thank you.

Otherwise, I would have had to recreate that probably all from scratch because I am not that tech savvy.

I am a little bit incompetent in terms of technology, despite my age.

So I get made fun of all the time by my students actually for that.

So but um uh I will probably uh are there any questions regarding the um presentation as it stands currently or any thing that um um council members or community advisors had for me regarding any changes or updates because otherwise I will probably come back at a future um at our next meeting with some suggested changes to this.

Otherwise, the bulk of the the the bones of this are not changing.

A little bit of the flesh might change a little bit here and there, but the bones of this are fantastic, and that's um kudos to um Mr.

Farnsworth for that.

I'm sad to see that he's leaving, so um my uh trying to think here.

I didn't see anything on here that um really won't warranted my asking questions or anything or that um anything of that sort.

Was there anything that any of you had questions about regarding this at this point in time?

I just had a question um on slide five where it says the first bullet 1990s audits.

I just don't know what that's regarding on YouTube.

I'm sorry, I didn't mean to interrupt you.

No, yeah, it's just um I don't know that piece of the history, so just curious.

Follow up back with former um chair of this committee, um council former council member Prince on that actually, because I know that she has a lot of institutional knowledge regarding how this committee was first uh how how it came into being, how it was created, because there used to be from my understanding from my conversations with her and Madam Chair, please correct me if I am wrong.

That uh during the 90s, there was a legislative audit program here in the council that was part of council research, and it was not a large team, but they did quite a bit of work over a period of time, and then uh there were budget cuts that basically came through from a particular mayor, and so um that were pushed pretty hard, and that whole program went away.

And so in 2018, um former council member Prince and Chair Necker, they um pretty much led the recreation of an of a basically a legislative audit activity here at City Hall.

So and I love it.

Love it.

But um is that is do I have the gist of the history correct, madam chair?

Yes, that's exactly right.

And I think what this prints could provide you with is some of the specific examples of the audits they did in the 90s, which um were just as juicy as the ones that we do today, I believe.

Yeah.

Ms.

Vang.

Yep.

I just want to share that those audits that were previously conducted in the 1990s are uploaded to our uh audit committee website for historical references.

And they were indeed done in-house because at the time council was a full-time council and a full staff.

So they were able to do uh have that level capacity to do that level study.

Yeah.

I've read some of those audit reports, they're delightful.

Yeah.

Um other comments on the presentation itself.

The only thing I would I would suggest is just that I think as you're talking about the former the audits that we've completed, um, just to give folks a sense of what came out of those, like what maybe some of the specific recommendations were, and especially ones that were actually followed up on.

You know, we talk often here about like while we did the study, what actually changed, and I think there are concrete examples of how departments have changed things, including by the way, just so everyone knows, um, Councilmember Coleman and I were at the CIB committee meeting last Monday, and they referenced um we were there because the city council was talking about their work as part of our budget process, and have been more interested in learning more about their work as part of our budget process and also because of our audit.

Um they welcomed us to their committee meeting, and while we were there, they talked, they actually had a report, not because we were there, just part of our normal agenda from the the commission member who is going to be leading the work on taking our recommendations uh forward.

So without us, no influence from us, that's just happening on its own, and I just thought people would like to know that.

So having some examples of like what actually changes on the ground because of these audits, I think is a maybe a good addition to the presentation.

Duly noted.

I'm gonna move us along just because unless there's any other comments on this, because I know we have just 12 minutes left.

Yeah.

Um I think for sake of time, topic selection process, I'm gonna just hold on because I think we've got it written down, we know it.

I don't know if there's a lot to discuss there.

No, I think it's really related to what um community advisor at um McVeigh just talked.

Yeah, and this is just kind of our broad outline of our process, and it doesn't deviate too much from the way we've always done it, but it's but is outlined here.

Great.

Um excellent.

And that's a good piece to share when you're presenting to these district councils as well.

Yeah, yeah.

Okay.

Great.

So then with that, we do have uh we need to talk about our community advisor appointment timeline.

I don't know if it was explicitly said, but Mr.

Frontworth unfortunately has moved across the river.

Yes.

So he is still alive and well and wonderful, but legally can always be.

So uh we do have an opening.

Um sorry, do you want to uh walk us through the timeline?

Yep, thank you, Chair Naker.

It's very similar to the last timeline that we used to seat um community advisor McVeigh to the audit committee.

So it's really quick because it's two-week posting.

And then we would then have two weeks to kind of review, rank our choices, and then invite our top three candidates for an interview or top two, and then we go through the process of making our um final selection and proposing that to the city council to vote on the matter, and then hopefully we can see the new member um in January.

Um but given that the holiday, and I wasn't sure if the committee would contemplate um extending the two weeks to maybe possibly four weeks so that it brings up into the new year, um, given that Thanksgiving, Christmas, and possibly New Year are in the mix.

We may not have as much of um as many qualified individuals that we we as we would normally hope for during the normal season.

So that's this is it.

What you see for you is the um application.

It doesn't change much from the last time.

Um we kept the question very similar um from the last time when we uh had a vacancy.

If you wish to change those questions now, we can possibly change it as well.

Um I thought they're very simple.

We didn't um we did away with the resume cover letter, but we wanted to give folks the option if they wanted to submit that for the audit committee to consider, and not everybody takes advantage of that, and that's fine because we eliminated that part.

And then um a press release will be ready to go out tomorrow.

The proposal is that this goes out tomorrow.

And um, if you would like to submit a quote for us to release with the press release, I would certainly welcome that as well.

But the goal is that this um application will be live tomorrow and the answer December 3rd, so that I would have enough time to compile the applications and then provide that information to the committee to review and rank.

So we'll work in within a very short time frame.

I would just uh off the bat suggest that we do have a longer posting.

I think two weeks with the holidays is not probably enough.

So I think it's all right if we have a little bit more time before we have the seat filled, but I would say just a four-week committee, Ms.

Coleman.

Yeah, thank you.

I'm just trying to think this through because in an ideal world, I feel like we would have the first two weeks in January to have people apply.

Obviously, that's much much longer, but you know, just thinking about when people are thinking about taking on new commitments in the cycle of the year.

Um at a minimum, yeah, I would certainly support keeping the posting open for longer.

I think having it open primarily over Thanksgiving week is just the the millions of applicants that I trust we would get if we posted it for a little bit longer.

Okay.

And then ideally, I think if we could line up the interviews with an existing meeting, and maybe we just have to plan for a little bit of a longer meeting that day.

I think that would be better than scheduling a different time for us all to come back together just to do interviews.

Looking to my colleagues on to see if I'm right about that or not.

We could also consider having a meeting in February.

Well, depending on, I guess it depends on where things land after you do the four-week extension or four-week posting.

Um I guess that would be moving us towards the end of January to have somebody seated, right?

Right if we did four weeks.

So we would, and because we wouldn't have a February meeting unless the committee deem it necessary, then we won't have an interview until March.

So maybe we should plan on the minute.

Sorry, it's going on the interview on January 20th, and at our meeting then I would let me pull up my calendar.

Give me a second.

I was just gonna ask, does this probably not well?

So did it does the interview have to take place at a meeting?

Okay.

Yeah, unless the committee would like to meet in a special meeting, then we'll find some additional time.

But if we go four weeks, then it's due Christmas Eve, the application.

Oh, let's not do that.

Unfortunately, I won't be working on Christmas Eve.

Um, and then um if we extend for five weeks, it brings us to New Year's Eve.

So then I don't know if you would like it to extend for seven weeks.

So that it would brings the application would then be due at January 7th.

We have a meeting on the 20th.

Would that give you enough time between the seventh and the ideally I would like a week to contact all the potential candidate of when we would meet, and usually that takes a little bit of finagling.

And you should dictate a little bit of finagling.

Would we thank you?

Um would we be able to have access to the applications on a rolling basis?

Um to be quite honest, you should you can, but we normally don't get a slew of application to the very end.

Unfortunately, that's how it's worked here.

The pattern hasn't changed too much.

Yeah.

That makes sense.

And we would need uh obviously a resolution that goes to council to vote on this new uh member before he or she can be seated.

Well, I think my recommendation would be that we make it due on the 5th, January 5th.

And then we have two weeks and a day before we would need to meet and interview the person, the people on January 20th.

Um I think if we're all on our game, and we usually don't get millions of applications as we know, we get a few, we can look at them.

We often I think interview most of the people who apply, so it's not like a hard decision as to who we invite.

And I think Ms.

Bang, we've done this in the past where we've told people on the application this is when you should be prepared to interview so they know.

Um January 20th at four or whatever.

Um then we can have the person seated by the March meeting.

Okay.

That'd be fine.

Yeah.

Um if the committee feels that that that's an appropriate timeline, I will certainly make the changes.

How do folks feel about doing sort of a I'm not this may not be a good idea, but I'm thinking we're we're all gonna be promoting the topic selection survey.

And we also want to be promoting the fact that we have an opening.

Um does it make sense to do sort of a combined audit committee posting, you know.

Hey everyone, two great opportunities.

One to tell us your ideas, and two to join us.

Like I'm just trying to think through because I don't otherwise we're doing two postings at the same time about the same thing, sort of, and maybe we should draft a blurb that kind of incorporates both.

Okay.

What what do others think?

I would support that.

They might actually garner um more attention, actually.

The way people's brains are wired, having that out there.

It sounds a little counterintuitive, but there is some research that suggests that when you have um two when you have a when you have when you release information as like a governmental entity and you have two pieces of like two storylines simultaneously commingling with one another, it actually can garner more attention because there's just more information being presented, basically.

That is counterintuitive, but fascinating.

It's a lizard brain thing.

Thank you.

Okay.

Well, let's do it if there's no objections.

So we'll look for a revised blurb for both topic selections and this process thing, and then um, I think maybe we could wait a wait to post until I mean we could give people some time to submit quotes, for example, maybe do it at the end of this week instead of tomorrow, just because now we're giving people a lot more time to apply.

Sure.

I would okay.

So then ideally Friday would be my due date to, or would you like next Monday?

Okay, great.

Okay.

All right.

Anything else to come before us?

Looks like we're at the end of our formal agenda.

Anything else you need from us today?

All right.

Well, thank you all for our productive meeting.

We got a ton done, as always.

Um everyone has a wonderful Thanksgiving.

So we next to each other, and we are journal.