David Moscatelli: Organizations are hesitant about public AI due to privacy concerns, local AI solutions are preferred in banking and healthcare, and the Go One device enhances on-premises AI scalability

Key Takeaways

• Organizations are increasingly hesitant to use public AI providers due to data privacy concerns.
• Go Abacus focuses on providing AI infrastructure tailored for regulated industries.
• Banks and healthcare institutions prefer local AI solutions to maintain data privacy and control costs.
• The Go One device enables on-premises AI by connecting to employee PCs and preconfiguring software.
• The Go One device supports up to 2,000 concurrent users and can be scaled by daisy-chaining multiple devices.
• Sentry’s AI debugging agent uses comprehensive data to identify root causes and suggest fixes.
• Stress testing has significantly lowered the chances of system failure in Go Abacus’s solutions.
• The Go One OS uses specialized language models trained on client-specific data for deterministic tasks.
• Client models are trained during off-hours, with updated weights sent back nightly.
• LLMs (Large Language Models) are essentially CSV files with weights and software to run those weights.
• Go Abacus’s approach highlights the importance of privacy and cost control in AI deployment.
• The scalability and flexibility of the Go One device make it suitable for large organizations.
• Go Abacus’s solutions are designed to enhance software reliability and developer efficiency.
• The company’s client-centric approach ensures adaptability in machine learning operations.
• Understanding the structure of LLMs demystifies their operation and utility.

Guest intro

David Moscatelli is founder and CEO of Go Abacus, a company building secure, on-premises AI infrastructure for regulated industries like banking. He has achieved significant early traction with the Go1 device, a $250,000 box that enables banks to run AI models without sending data to cloud providers, and has already secured 1,600 pre-orders while hitting $1M in annual recurring revenue.

Why organizations are hesitant about public AI providers

• Organizations are hesitant to use public AI providers due to data privacy concerns.

  — David Moscatelli

• Banks, credit unions, and hospitals are wary of sending data to public AI providers.

• They are not interested in sending their data to any of the public AI providers.

  — David Moscatelli

• Concerns about data leaving their infrastructure drive the need for local AI solutions.

• How do we use AI but locally within our network so none of our data leaves our infrastructure?

  — David Moscatelli

• Privacy concerns are a significant barrier to AI adoption in sensitive sectors.
• The trend is moving towards localized AI solutions to address these privacy concerns.
• Understanding these concerns is crucial for AI deployment in regulated industries.

Go Abacus’s focus on regulated industries

• Goabacus aims to provide AI infrastructure specifically designed for regulated industries.

  — David Moscatelli

• The company targets a niche market with specific needs for privacy and compliance.

• We’re AI infrastructure for regulated industries.

  — David Moscatelli

• Understanding the challenges faced by these industries is key to Go Abacus’s business model.
• The focus on regulated industries highlights the company’s market positioning.
• Go Abacus’s solutions are tailored to meet the compliance requirements of these sectors.
• The company’s approach addresses the critical need for secure data management.
• Go Abacus is at the forefront of AI deployment in sensitive environments.

Local AI solutions for banks and healthcare

• Banks and healthcare institutions prefer to use AI locally to maintain data privacy and control costs.

  — David Moscatelli

• Local AI solutions help institutions avoid sending data to external providers.

• How do we use AI locally within our network?

  — David Moscatelli

• Cost management is a significant factor in the preference for local AI solutions.

• You want a fixed price.

  — David Moscatelli

• Understanding these preferences is crucial for AI providers targeting these sectors.
• Local solutions address both privacy and cost concerns effectively.
• The trend towards local AI solutions is growing in the banking and healthcare industries.

The Go One device and its capabilities

• The Go One device enables on-prem AI by preconfiguring software and connecting to employee PCs.

  — David Moscatelli

• The device is designed to facilitate AI access within organizations.

• We have a piece of hardware which is the Go One.

  — David Moscatelli

• The Go One device supports up to 2,000 concurrent users.

• That device can support up to 2,000 concurrent users at one time.

  — David Moscatelli

• Scalability is achieved by daisy-chaining multiple devices.

• You can chain them together to four six eight etcetera.

  — David Moscatelli

• The device’s scalability and flexibility make it suitable for large organizations.
• Understanding the technical setup of the Go One device is crucial for enterprise environments.

Enhancing software reliability with AI debugging

• Sentry’s AI debugging agent uses comprehensive data to identify root causes of problems and suggest fixes.

  — David Moscatelli

• The agent enhances software reliability and developer efficiency.

• Sentry’s AI debugging agent uses all this data and context.

  — David Moscatelli

• Efficient problem resolution is crucial in software development.
• The debugging agent’s capabilities highlight the role of AI in software maintenance.
• Understanding the importance of efficient problem resolution is key for developers.
• The agent’s use of comprehensive data ensures accurate problem identification.
• AI debugging tools are becoming essential in modern software development.

The importance of stress testing in AI systems

• The chances of failure in our systems are incredibly low due to stress testing.

  — David Moscatelli

• Stress testing ensures system robustness and reliability.

• On the basis of our stress testing the chances of failure is incredibly low.

  — David Moscatelli

• Understanding stress testing methodologies is crucial for technology providers.
• Stress testing is a critical component of system reliability.
• The effectiveness of stress testing is emphasized in Go Abacus’s solutions.
• Ensuring system robustness is essential for AI deployment in sensitive environments.
• Stress testing methodologies impact the reliability of AI systems significantly.

Specialized language models in the Go One OS

• The Go One OS uses a collection of specialized language models that are trained on client-specific data.

  — David Moscatelli

• The models perform deterministic tasks in banking and other sectors.

• Each of those models specializes in a very specific task.

  — David Moscatelli

• The use of specialized models highlights the OS’s unique architecture.
• Understanding the structure of the Go One OS is crucial for its application in banking.
• The models’ specialization ensures efficiency in performing specific tasks.
• The OS’s approach to model training is client-centric and adaptable.
• Specialized models are becoming increasingly important in AI applications.

Client-centric model training in the Go One OS

• The company trains its models on client data during off-hours and sends the updated weights back to the clients nightly.

  — David Moscatelli

• This approach ensures the models are always up-to-date with client data.

• We batch train on their data and then we take the weights from that training.

  — David Moscatelli

• The client-centric approach highlights the adaptability of the Go One OS.
• Understanding how client data is integrated into model training is crucial.
• The nightly update of weights ensures the models remain relevant and accurate.
• This approach emphasizes the importance of client-specific data in model training.
• The Go One OS’s adaptability is a key feature for its application in various sectors.

Demystifying LLMs and their structure

• An LLM is essentially a CSV file with weights and software to run those weights.

  — David Moscatelli

• This explanation simplifies the understanding of LLMs.

• Hate to demystify for everyone that’s what it is.

  — David Moscatelli

• Understanding the structure of LLMs is valuable for their operation.
• The explanation highlights the fundamental components of LLMs.
• Simplifying the concept of LLMs aids in their application and understanding.
• The demystification of LLMs is crucial for their broader adoption.
• Understanding the operation of LLMs is essential for AI practitioners.