Small+Systems+Forum+at+ACE+2012

**Discussion Questions for Forum on** **Small Systems Information Dissemination Strategies**
 * Question 0: What can we do to get information to small systems?**

Strategies include: i. Hired by systems, financial incentives can be issue. ii. Not out there educating small systems, only responding to needs, working with those paying them. iii. Small communities are at their will for infrastructure projects (grant writing, project scope)
 * 1) Classes/meetings
 * 2) Credit classes/workshops
 * 3) Conferences
 * 4) Webinars
 * 5) Distance learning/grouped webinars
 * 6) Operator groups
 * 7) Direct assistance
 * Engineers
 * Engineers

3. Regulators

i. Varies dramatically from state to state ii. Some states have trained staff to direct assist or train iii. Some are strictly regulators, onus is on system

4. TA providers i. Trainers ii. Circuit riders iii. Referrals
 * 1) RCAP/RW – generally are operators themselves
 * 2) RCAP/RW – concentrate on small systems (funding requirements)
 * 3) AWWA/WEF – staffs are not operators, maybe engineers in some sections, but not others
 * 4) AWWA/WEF – use consultants/engineers who are generally members for teaching classes
 * 1) NRWA has a network of staff, who are operators, who go out to facilities, work with operators, assist with problems, perform services like leak detection, generally operations oriented
 * 2) RCAP has some staff who are operators, who go out to facilities, generally by request, not as part of routine workload, but also assist with problems, generally more financial and managerial oriented
 * 1) Peer operators

i. Operator referrals ii. Pilot efforts i. RCAP has developed a lot of great free information for operators, decision makers and small system users.
 * 1) Online/Print resources
 * 2) Regulatory agencies
 * 3) TA providers

3. Vendors 4.Consultants 5.Member organizations i. Wide differences in cost, AWWA being expensive, (WEF and ASCE are comparable to AWWA) Is NRWA less expensive? State sections are definitely less expensive

ii. Some require membership to see/use resources (WEF recently started moving to making some materials available to anyone)
 * 1) Marketing Campaigns (Can we get an example here? Are these vendor campaigns?)
 * 2) Generally don’t reach small systems
 * 3) Plenty of materials, but not enough effort to engage audience to use
 * 4) For small systems, a “secondary” priority (or less), need to develop understanding of value, don’t have time or money for marketing


 * 1.Can you identify a strategy (or strategies) for providing information to small systems that you feel is particularly effective (including consideration of cost) and merits greater attention and use? **

Small systems are often use dedicated consultants. Hardcopy manuals are valued by these consultants and appear to be an effective means of information dissemination. 94% of consultants report using these, the highest rating of any information pathway included in the survey of Yuel et al. (2010). They were also the most preferred source of information in that survey.

(a) Pilot- and full-scale demonstration programs and sites fully or partially funded by EPA. These include treatment technologies, condition assessment and rehabilitation technologies, and green infrastructure technologies; pathways exist to get this information to small systems but further development or alternative approaches may merit consideration, (Can we get some citations here? any history that say EPA piloted technology X and it was adopted?)

(b) Workshops funded by EPA that are geared toward small system utilites, owners and operators, and regulators; and

(c) webinars (from EPA? other providers?) that are free to the public.(Note that we have AWWA webcasts as a less effective approach in Q#2 below. And AWWA webcasts are NOT free. IS there evidence that free really makes the different?)

Nebraska uses some of its 2% SRF set aside to fund a contract with NeRWA.

an informal and cooperative group called the Nebraska Training Coalition consisting of the 6 primary operator training providers in Nebraska, one of which is NeRWA.

The face-to-face circuit rider concept of State Rural Water Associations and other organizations representing small water systems has demonstrated some effectiveness. Contracts for technical assistance, with contracts specific to the dissemination outcomes desired, should be utilized. The effectiveness of this may be due to ongoing relationships with time. This strategy does have cost associated with it.

Ongoing relationships with small systems operators are particularly effective. The success of NRWA may not just be due to the in person nature but the ongoing interactions.

Smallwatersupply.org maintains a calendar of webinars and in-person trainings.

EPA ORD conducts an annual conference for state drinking water administrators. Funding for travel is no longer available. (What percentages of states are typically represented?)

Based on survey results, operators of small systems do appreciate the opportunity to escape the small water system and network with others on a limited time basis. One-day seminars outside of their day-to-day confines are highly rated.

Montana Water Center developed CDs that are still in use though some are becoming dated as this center is no longer able to produce these materials.


 * 2. Are there any activities that have not been effective or successful as implemented and might serve as useful lessons learned? **

AWWA webcasts received the low ratings in the survey of Yuel et al. (2010). These may be too expensive or not targeted at the right topics for small systems.

WEF has made webcasts free. EPA offers free webcasts but notices to do get to small systems.

Dissemination of information solely by electronic means to small water systems has not been particularly successful. Timing of information dissemination can also be critical – too early and it will be set aside; too late and there will be no time to react. Also, time of year is a factor. Scheduled free time from the spring season through fall can be limited. Computer usage by operators of very small systems can be limited; not particularly due to abilities or access, but due to time. They wear many hats and their time spent on actual water system duties would not normally be full time – it is split between water, wastewater, streets, mowing the parks, hanging up Christmas lights on main street, etc. In some areas, the demographics of individuals first entering the water industry have changed. This is driven by the economy and a somewhat more mobile society. (Please explain old demographics and new. What is the difference?) (These judgments definitely reflect small water system observations. The state in which I work has 746 community and non-transient non-community public water systems. Of these, 29 serve greater than 5,000 people, and 14 of those 29 serve greater than 10,000 people. As such, these small water systems have minimal personnel with many of the smaller systems having only one utility employee.)

Lengthy conferences (more than a couple of days) are not necessarily effective due to lack of attendance by operators of small water systems.


 * 1) Large marketing campaigns from AWWA and WEF develop materials but put [large] systems in charge of legwork to implement, so most small systems don’t participate or are not members and don’t even know about it.
 * 2) Many print resources from USEPA and others never make it into the hands of small systems, there is no mechanism for letting systems know about these resources or how to go about getting them.
 * 3) It all comes back to small systems typically need face-to-face assistance or at least direct contact to get things to them.


 * 3. What evidence (peer reviewed literature, experience, etc.) exists to evaluate small system dissemination strategies? What is the basis for your judgments in 1 and 2 above? **

Yeuell, H.D., P.L. Gurian, B.I. Dvorak, and D. Lytle. 2010. “Improving Information Dissemination to Consulting Engineers Serving Small Systems: Suggestions Based on National Survey,” AWWA 2010 Annual Conference and Exhibition, Chicago, IL.

Mike's NE paper.

Joy may have some insight.


 * 4.What metrics and benchmarks may be helpful in evaluating the performance of small systems dissemination efforts? Given the complexities in how information is disseminated, where may efforts to measure performance be problematic? **

Survey ratings can be helpful but are only qualitative impressions.

outputs and outcomes from federal funding are reported by awardees, RCAP will report metrics, Rural Water has summarized how many people helped. Maria may have some EPA info on this subject. On the operational side we could try to look at operator skill gains/$. It would be more feasible to measure contact hours/$.

CDs/materials for trainings developed per $

Then major capital investments and regulatory compliance issues have a different set of actors and information needs. We could look at small system consultant contact hours/$. Also state regulator contact hours/$.

If dissemination of information is of a regulatory nature, data on compliance could be obtained. (This implies a longterm study of the desired outcome vs. training. This seems to make incredible sense but has it been done???)

Another means could be the distribution of “homework” in some fashion. This would obviously necessitate a repetitive effort on one topic/issue. (So the benchmark would be "correct answers"? Sort of a standardized testing for operators? How standardized are the certification exams and can performance on these exams be a benchmark?)


 * 5. Where might research efforts be focused to improve small system dissemination strategies? What additional knowledge could feasibly be obtained by research and lead to improved knowledge dissemination? **

I don't know of any effort to assess contact hours/$ across different strategies. I am also not aware of skill increases being measured by different training formats (internet synchronous, internet asynchronous, site visit, group training, etc.). A needs assessment based on both what operators want to learn and what regulators feel they need to know could help to identify topics that would be relevant. Relevancy is a key issue according to Yuel et al.

The effect of training on compliance. Is anyone aware of a study correlating the two? Maybe a case-control type design where the violators are the cases? We would then have to carefully match controls so this might be a tricky study to do right.

Better understanding of demographics of audience. (PLG thinks this may really be a need for more pilot testing of messages and modules. Different demographics may have different needs but we may not know what works just on demographics.)


 * 6. What actions are recommended? **

As an outcome to EPA’s 2011 Small Drinking Water System Workshop, we formed a workgroup to address EPA’s communication with small systems. The general consensus was that EPA's communication should be proactive not reactive, for the communication to provide an explanation/reason for a new rule/regulation, and for the communication to be targeted, taking into account the different needs of the operators. We agreed at the workshop to develop an outreach product with input from the whole group in hopes of optimizing the value and usefulness of the information. It is my hope that once the developed communication is disseminated, we will be able to document its effectiveness (assess whether or not there is relevant and useful improvement/gain in the knowledge of alternative treatment technologies, sampling techniques, test procedures, sources of contaminants).

Develop testbed for communication materials.

Link compliance data to information needs to dissemination network. Research information architecture needed and message format needed. Tried on LTESWTR2 but got some bounce back. But if you reach them it can be really helpful.

EPA has internal data on arsenic hits. States may have more detailed info on systems and their characteristics and challenges.

The decision makers (boards/councils) of small water systems need to become more aware of their responsibilities relative to the water system. Often, the water system as viewed by the small town board consists only of those items visible – fire hydrants, elevated water storage and water rates. (So the action item is to train these people? I think RCAP has some experience in doing this.)