Vast Scientific - Boundless Informatics Solutions for Science
strategic assessment services

VAST Scientific provides independent, analytically-rigorous evaluation of an emerging technology system throughout its development stages.  At early stages, strategic assessment ensures that the system under design is focused on meeting identified industry capability needs and is adoptable by users into their operating environments.  At later stages, measurement-based assessment reports provide developers, financiers, and other stakeholders with defensible knowledge on the current status of the venture and of the issues requiring remediation to meet venture goals. 

Modern information-driven systems are complex, involving integrated commercial-off-the-shelf (COTS) and private source software components supporting data management and information extraction across a distributed environment of privilege-managed shared services and products that must meet the needs of a diverse user population.  While components may be rigorously tested prior to integration, it is essential to thoroughly test their performance in the context of system operations.  This provides incredible insight into performance bottlenecks and their impact on the effectiveness of the system to its users.  The system must also be assessed under realistic deployment strategies and user loads.  While common practice, it is inadequate to use limited functional demonstrations as a measure of venture status. 

We propose that an integrated and comprehensive assessment strategy can streamline the development process by ensuring capability benchmarks are met for a realistic operational environment.  Strategic assessment services include:

  • Research system capabilities are matched to documented industry needs, and propose technology candidates that can support these capabilities (initial architecture support).

  • Decompose the proposed system architecture into its key functional components to identify a comprehensive suite of metrics (measures of performance and measures of effectiveness) needed to perform a thorough assessment.

  • Design a data collection and analysis plan that supports the metric assessment, including the operating environment and user interactions with the system.

  • Identify key system data collection points for each functional component and coordinate with developers to insure appropriate data capture instrumentation.  As a leave-behind, this can provide a long term benefit to the administrating agency, who can continue to mine these system data feeds for component or node performance, user usage, and resource management beyond the assessment stage.

  • Execute assessment data collection events in a real-world or representational environment, including the capture of subject matter expert opinion on apparent performance (lessons learned).

  • Analyze system data against measures of performance and assess system effectiveness by comparing performance to user and community expectations.

  • Report assessment findings coupled with recommendations for remediating identified issues.  Assessment findings are then used to drive the next development cycle. 

To promote effectiveness, it is essential that VAST Scientific assessors work closely with system developers.  Strategic assessment is not antagonistic in nature, as it is designed to document the successes as well as the issues requiring remediation, and includes the expert opinion of the developers.  However, objectivity is maintained through its basis in scientifically defensible measurements obtained through our third party testing and analysis. 
If objectivity is not required, we can design an evaluation plan that our customer can then execute themselves.

[ back to top ]