We are talking a lot about Artificial Intelligence and Machine Learning.
Most of us know about OpenAI and their #ChatGPT which helps a lot with many tasks. However this is only the tip of the iceberg.
You have to know your “enemy” before you start fight on the #IT hashtag#OT Cyber Field. AI itself is a big risk from the hashtag#cybersecurity perspective and to get better understating about it we have to check under the hood.
But where to start?
There are many options like technical documentation, cloud computing but let’s find something achievable and open source.
I started my AI journey with building Workstation based on the dual Intel Corporation Xeon E5-2699v4 (total 44 cores / 88 threats), running on the Supermicro mainboard (IPMI is great thing), 512 GB DDR4 ECC Ram and dual NVIDIA Tesla P100 16GB HBM2.
I couldn’t make it run without #3D printed cooling adapters for the CPU’s and GPU’s.
You can also use other platforms like AMD hashtag#EPYC with #Radeon #Instinct GPU which are also great for this purpose.
Next thing was installing Canonical #Ubuntu LTS running on bare metal.
You may try using virtualization like Proxmox Server Solutions or #NXP-NG and then Ubuntu 😉
Let’s go to AI Software.
1. ChatGPT like – Ollama
First step was to install Ollama and WebUI which gives you well known interface to act with the application. Then we need do download models – this is more tricky because depends how much GPU memory you have, that model may be best for you.
For example LLAMA 3.1 8B from #Meta require :
Higher Precision Modes
32-bit Mode: ~38.4 GB
16-bit Mode: ~19.2 GB
Lower Precision Modes
8-bit Mode: ~9.6 GB
4-bit Mode: ~4.8 GB
You can run bigger models if you have less memory, but will consume RAM which is slower than vRAM (believe me I tested 405B which took 231 GB RAM + 32 GB vRAM.
Of course you can try with different models (even uncensored).
2. ComfyUI – Stable Diffusion Image generator
To make it run, you need to install it and get some models. You can find many of them available on specific sites or train your own. For the test purpose I downloaded free from the #CivitAI.
ComfyUI require GPU processing + memory, however was not able to utilise two Tesla’s at once yet.
Was able to generate very realistic images which get me think that is so easy to create fake news, fake photos, or great #phishing campaigns.
So far I’m still testing and playing around but I’m getting better understating about the AI and LLM’s.
Can see many risks it brings to IT and OT world, we need to face.
Share with me your thoughts and experience with AI, I’m looking forward for some inspiration and ideas.
Links:
https://ollama.com/
https://lnkd.in/dj-Q4Swj
https://lnkd.in/dhWGcE6i
https://civitai.com/