One reason ISC was excited to lead the scoring and evaluation portions of the CA Port Data Interoperability Grant Program was that it represented an opportunity to identify and implement improvements to the process. In typical government grant applications, the opportunities for applicants to engage with the granting agency is limited. There’s a period in which applicants can ask questions, which is valuable in its own right. But so far as we know, there has never been a grant program that gave its applicants feedback, and the opportunity to make changes to their proposals, during the application process.

Of course, we were able to do this partially because the candidates were known and limited (the grant program is only open to the five containerized ports in CA). If this grant had been open to the public, with an unknown number of candidates, this would have been harder to execute. Still, we hope the positive feedback we received from our stakeholders about this design change will inspire opportunities for feedback during future grant applications. We strongly feel that this addition to the process allowed the proposals to be stronger, more competitive, and more aligned with the goals and budget of the State of CA.

Here is an overview of the steps in our application process:

1. The scoring criteria were crafted alongside the original proposal guidelines after hosting workshops with the ports to learn more about their data needs and individual visions.
2. Technical Advisory Committee (TAC) members reviewed the scoring criteria before reading draft proposals, to ensure A) consistency with what we instructed the ports to write, and B) that we were making reasonable suggestions.
3. To keep things streamlined, individual TAC members reviewed each proposal, focusing on high-level project vision, scope of work, and implementation details in their respective area of expertise. Crucially, we did not score the proposals in this phase, not only to be efficient with our project budget, but also because we didn’t want the ports to optimize their final proposals to match an early score.
4. As they reviewed the individual documents, TAC members added comments inline with the text. Those documents were then shared back to the ports.
5. We reviewed each proposal as a group, in an interactive process facilitated by ISC, where we provided feedback in three main areas:
   • Things we liked
   • Things that needed improvement
   • Suggested changes in direction or scope
6. At the end, we stepped back and explored the trends we noticed across all proposals, focused on common areas for improvement.
7. We voted on one proposal to run through a mock scoring session, to exercise our process ahead of time.
8. We summarized the feedback for each port, starting with the trends we observed across all applications (we sent these trends to all applicants). Then, we summarized the comments specific to individual proposals (generated during the group process) and sent them to each port respectively.

This process was well-received by the ports, and we observed several positive ripple effects as a result of the feedback. In general, the proposals improved substantially between the initial and final submissions. Because the TAC is already experienced with the technical aspects of data interoperability, we asked the applicants to remove unnecessary explanations of technical concepts and instead focus on the implementation details. The revised proposals focused more on how they would accomplish what they set out to do, rather than a nonspecific vision statement with few details. We believe that the resulting applications were much more detail-oriented, and much more aligned with the goals of the State of CA, after the feedback and revision process. Additionally, the feedback process allowed us to:

• Identify opportunities for collaboration between ports
• Request that ports demonstrate how they’ll integrate existing standards (mostly DCSA standards) into their proposed project
• Identify how ports can use this funding opportunity as leverage for future funding opportunities

We are grateful for the opportunity to implement this new kind of process, and we’re excited to be a part of the future of data interoperability in this critical part of our national supply chain. Shoutouts to all of our partners on the project: GO-Biz, Momentum, Port of LA, Port of Long Beach, Port of Oakland, Port of San Diego, Port of Hueneme, DCSA, CAPA, Latacora, Data CRT, and Cloud303.