Current File : //www/varak.net/nextcloud.varak.net/core/doc/admin/ai/app_recognize.html
<!DOCTYPE html>
<html class="writer-html5" lang="en" data-content_root="../">
<head>
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>App: Recognize — Nextcloud latest Administration Manual latest documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../_static/css/theme.css?v=19f00094" />
<link rel="stylesheet" type="text/css" href="../_static/copybutton.css?v=76b2166b" />
<link rel="stylesheet" type="text/css" href="../_static/dark_mode_css/general.css?v=c0a7eb24" />
<link rel="stylesheet" type="text/css" href="../_static/dark_mode_css/dark.css?v=70edf1c7" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script src="../_static/jquery.js?v=5d32c60e"></script>
<script src="../_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
<script src="../_static/documentation_options.js?v=c6e86fd7"></script>
<script src="../_static/doctools.js?v=888ff710"></script>
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../_static/clipboard.min.js?v=a7894cd8"></script>
<script src="../_static/copybutton.js?v=f281be69"></script>
<script src="../_static/dark_mode_js/default_light.js?v=c2e647ce"></script>
<script src="../_static/dark_mode_js/theme_switcher.js?v=358d3910"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="App: Context Chat" href="app_context_chat.html" />
<link rel="prev" title="App: Local Whisper Speech-To-Text (stt_whisper2)" href="app_stt_whisper2.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../contents.html">
<img src="../_static/logo-white.png" class="logo" alt="Logo"/>
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../index.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="../release_notes/index.html">Release notes</a></li>
<li class="toctree-l1"><a class="reference internal" href="../release_schedule.html">Maintenance and release schedule</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installation/index.html">Installation and server configuration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration_server/index.html">Nextcloud configuration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../apps_management.html">Apps management</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration_user/index.html">User management</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration_files/index.html">File sharing and management</a></li>
<li class="toctree-l1"><a class="reference internal" href="../file_workflows/index.html">Flow</a></li>
<li class="toctree-l1"><a class="reference internal" href="../groupware/index.html">Groupware</a></li>
<li class="toctree-l1"><a class="reference internal" href="../office/index.html">Office</a></li>
<li class="toctree-l1"><a class="reference internal" href="../reference/index.html">Reference management</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Artificial Intelligence</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="overview.html">Overview</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_assistant.html">Nextcloud Assistant</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_translate2.html">App: Local Machine translation 2 (translate2)</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_llm2.html">App: Local large language model (llm2)</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_stt_whisper2.html">App: Local Whisper Speech-To-Text (stt_whisper2)</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">App: Recognize</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#front-end">Front-end</a></li>
<li class="toctree-l3"><a class="reference internal" href="#requirements">Requirements</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#disk-space-usage">Disk space usage</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#installation">Installation</a></li>
<li class="toctree-l3"><a class="reference internal" href="#scaling">Scaling</a></li>
<li class="toctree-l3"><a class="reference internal" href="#app-store">App store</a></li>
<li class="toctree-l3"><a class="reference internal" href="#repository">Repository</a></li>
<li class="toctree-l3"><a class="reference internal" href="#known-limitations">Known Limitations</a></li>
<li class="toctree-l3"><a class="reference internal" href="#ethical-ai-rating">Ethical AI Rating</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#rating-for-photo-object-detection-green">Rating for Photo object detection: Green</a></li>
<li class="toctree-l4"><a class="reference internal" href="#rating-for-photo-face-recognition-green">Rating for Photo face recognition: Green</a></li>
<li class="toctree-l4"><a class="reference internal" href="#rating-for-video-action-recognition-green">Rating for Video action recognition: Green</a></li>
<li class="toctree-l4"><a class="reference internal" href="#rating-music-genre-recognition-yellow">Rating Music genre recognition: Yellow</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="app_context_chat.html">App: Context Chat</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_summary_bot.html">App: Summary Bot (Talk chat summarize bot)</a></li>
<li class="toctree-l2"><a class="reference internal" href="app_api_and_external_apps.html">AppAPI and External Apps</a></li>
<li class="toctree-l2"><a class="reference internal" href="ai_as_a_service.html">AI as a Service</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../webhook_listeners/index.html">Webhook Listeners</a></li>
<li class="toctree-l1"><a class="reference internal" href="../windmill_workflows/index.html">Windmill Workflows</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration_database/index.html">Database configuration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration_mimetypes/index.html">Mimetypes management</a></li>
<li class="toctree-l1"><a class="reference internal" href="../maintenance/index.html">Maintenance</a></li>
<li class="toctree-l1"><a class="reference internal" href="../issues/index.html">Issues and troubleshooting</a></li>
<li class="toctree-l1"><a class="reference internal" href="../gdpr/index.html">GDPR-compliance</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../contents.html">Nextcloud latest Administration Manual</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content style-external-links">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../contents.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="index.html">Artificial Intelligence</a></li>
<li class="breadcrumb-item active">App: Recognize</li>
<li class="wy-breadcrumbs-aside">
<a href="https://github.com/nextcloud/documentation/edit/master/admin_manual/ai/app_recognize.rst" class="fa fa-github"> Edit on GitHub</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="app-recognize">
<h1>App: Recognize<a class="headerlink" href="#app-recognize" title="Link to this heading"></a></h1>
<p id="ai-app-recognize">The <em>recognize</em> app provides media tagging and face recognition functionality for the photos app. <em>Recognize</em> can group similar faces on user’s photos (“face recognition”); it can add fitting tags to photos detecting landscapes, food, vehicles, buildings animals and other objects, as well as known landmarks and monuments; it can recognize music genres in user’s audio files and adds tags for those; it can recognize human actions on user’s video files and add tags for them. It specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities.</p>
<section id="front-end">
<h2>Front-end<a class="headerlink" href="#front-end" title="Link to this heading"></a></h2>
<p>Tagged files will appear in the Photos app under the “Tags” section as well as in the normal Files app. Face recognition results will appear under the “People” section in the Photos app.</p>
</section>
<section id="requirements">
<h2>Requirements<a class="headerlink" href="#requirements" title="Link to this heading"></a></h2>
<ul>
<li><p>Nextcloud AIO is not supported but will likely work at sub optimal speed</p></li>
<li><p>Minimum supported Nextcloud version: 26</p></li>
<li><p>x86 CPU</p></li>
<li><p>GNU lib C</p></li>
<li><p>Background Jobs must be executed via cron</p></li>
<li><p>Using GPU processing is supported, but not required; slow performance is expected if you are not using a GPU</p></li>
<li><p>We currently only support NVIDIA GPUs</p></li>
<li><p>For GPU support you need to install:</p>
<blockquote>
<div><ul class="simple">
<li><p>NVIDIA® GPU drivers version 450.80.02 or higher.</p></li>
<li><p>CUDA® Toolkit 11.x</p></li>
<li><p>cuDNN SDK 8.x</p></li>
</ul>
</div></blockquote>
</li>
<li><p>GPU Sizing</p>
<blockquote>
<div><ul class="simple">
<li><p>The models used by recognize require about 1GB of VRAM or less</p></li>
</ul>
</div></blockquote>
</li>
<li><p>CPU Sizing</p>
<blockquote>
<div><ul class="simple">
<li><p>If you don’t have a GPU, this app will utilize your CPU cores</p></li>
<li><p>The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores</p></li>
<li><p>In the app settings you can set the number of cores to use</p></li>
<li><p>At least ~4GB of RAM dedicated for recognize</p></li>
</ul>
</div></blockquote>
</li>
</ul>
<section id="disk-space-usage">
<h3>Disk space usage<a class="headerlink" href="#disk-space-usage" title="Link to this heading"></a></h3>
<blockquote>
<div><ul class="simple">
<li><p>~1.5GB for all models in total</p></li>
</ul>
</div></blockquote>
</section>
</section>
<section id="installation">
<h2>Installation<a class="headerlink" href="#installation" title="Link to this heading"></a></h2>
<ol class="arabic">
<li><p>Install the <em>recognize</em> app via the “Apps” page in Nextcloud, or by executing</p>
<p>occ app:enable recognize</p>
</li>
<li><p>Execute the following command on your server terminal of each node that runs background jobs:</p>
<p>occ recognize:download-models</p>
</li>
<li><p>Go to your Nextcloud Administration settings and open the <em>recognize</em> admin settings page</p></li>
<li><p>Enable all modes of operation that you want the app to undertake</p></li>
<li><p>Enable GPU mode if you have a GPU that you want to use; if you want to use CPU only, you can set the number of cores to use here</p></li>
<li><p>Execute the following command on your server terminal to stop background processing of existing files:</p>
<p>occ recognize:clear-background-jobs</p>
</li>
<li><p>Execute the following command on your server terminal to process all existing files in bulk (This may take a long time, depending on how many files you have on your instance):</p>
<p>occ recognize:classify</p>
</li>
<li><p>Execute the following command on your server terminal to calculate face clusters from faces found in all existing files (Run this repeatedly until no more clusters are found):</p>
<p>occ recognize:cluster-faces</p>
</li>
<li><p>All new files from this point on will be automatically processed in background tasks without manual intervention</p></li>
</ol>
</section>
<section id="scaling">
<h2>Scaling<a class="headerlink" href="#scaling" title="Link to this heading"></a></h2>
<p>It is possible to scale this app by adding multiple “background” nodes to your cluster that will only process background jobs by executing cron.php.</p>
</section>
<section id="app-store">
<h2>App store<a class="headerlink" href="#app-store" title="Link to this heading"></a></h2>
<p>You can also find the app in our app store, where you can write a review: <a class="reference external" href="https://apps.nextcloud.com/apps/recognize">https://apps.nextcloud.com/apps/recognize</a></p>
</section>
<section id="repository">
<h2>Repository<a class="headerlink" href="#repository" title="Link to this heading"></a></h2>
<p>You can find the app’s source repository on GitHub where you can report bugs and contribute fixes and features: <a class="reference external" href="https://github.com/nextcloud/recognize">https://github.com/nextcloud/recognize</a></p>
<p>Nextcloud customers should file bugs directly with our Support system.</p>
</section>
<section id="known-limitations">
<h2>Known Limitations<a class="headerlink" href="#known-limitations" title="Link to this heading"></a></h2>
<ul class="simple">
<li><p>Make sure to test whether the functionality meets the use-case’s quality requirements</p></li>
<li><p>Machine learning models notoriously have a high energy consumption</p></li>
<li><p>Customer support is available upon request, however we can’t solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI)</p></li>
</ul>
</section>
<section id="ethical-ai-rating">
<h2>Ethical AI Rating<a class="headerlink" href="#ethical-ai-rating" title="Link to this heading"></a></h2>
<section id="rating-for-photo-object-detection-green">
<h3>Rating for Photo object detection: Green<a class="headerlink" href="#rating-for-photo-object-detection-green" title="Link to this heading"></a></h3>
<p>Positive:</p>
<ul class="simple">
<li><p>The software for training and inference of this model is open source</p></li>
<li><p>The trained model is freely available, and thus can be run on-premises</p></li>
<li><p>The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.</p></li>
</ul>
</section>
<section id="rating-for-photo-face-recognition-green">
<h3>Rating for Photo face recognition: Green<a class="headerlink" href="#rating-for-photo-face-recognition-green" title="Link to this heading"></a></h3>
<p>Positive:</p>
<ul class="simple">
<li><p>The software for training and inference of this model is open source</p></li>
<li><p>The trained model is freely available, and thus can be run on-premises</p></li>
<li><p>The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.</p></li>
</ul>
</section>
<section id="rating-for-video-action-recognition-green">
<h3>Rating for Video action recognition: Green<a class="headerlink" href="#rating-for-video-action-recognition-green" title="Link to this heading"></a></h3>
<p>Positive:</p>
<ul class="simple">
<li><p>The software for training and inferencing of this model is open source</p></li>
<li><p>The trained model is freely available, and thus can be ran on-premises</p></li>
<li><p>The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage.</p></li>
</ul>
</section>
<section id="rating-music-genre-recognition-yellow">
<h3>Rating Music genre recognition: Yellow<a class="headerlink" href="#rating-music-genre-recognition-yellow" title="Link to this heading"></a></h3>
<p>Positive:</p>
<ul class="simple">
<li><p>The software for training and inference of this model is open source</p></li>
<li><p>The trained model is freely available, and thus can be run on-premises</p></li>
</ul>
<p>Negative:</p>
<ul class="simple">
<li><p>The training data is not freely available, limiting the ability of external parties to check and correct for bias or optimise the model’s performance and CO2 usage.</p></li>
</ul>
<p>Learn more about the Nextcloud Ethical AI Rating <a class="reference external" href="https://nextcloud.com/blog/nextcloud-ethical-ai-rating/">in our blog</a>.</p>
</section>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="app_stt_whisper2.html" class="btn btn-neutral float-left" title="App: Local Whisper Speech-To-Text (stt_whisper2)" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="app_context_chat.html" class="btn btn-neutral float-right" title="App: Context Chat" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2024 Nextcloud GmbH.</p>
</div>
</footer>
</div>
</div>
</section>
</div>
<div class="rst-versions" data-toggle="rst-versions" role="note" aria-label="Versions">
<span class="rst-current-version" data-toggle="rst-current-version">
<span class="fa fa-book"> Read the Docs</span>
v: latest
<span class="fa fa-caret-down"></span>
</span>
<div class="rst-other-versions">
<dl>
<dt>Versions</dt>
<dd><a href="https://docs.nextcloud.com/server/28/admin_manual">28</a></dd>
<dd><a href="https://docs.nextcloud.com/server/29/admin_manual">29</a></dd>
<dd><a href="https://docs.nextcloud.com/server/stable/admin_manual">stable</a></dd>
<dd><a href="https://docs.nextcloud.com/server/latest/admin_manual">latest</a></dd>
</dl>
<dl>
<dt>Downloads</dt>
</dl>
<dl>
<dt>On Read the Docs</dt>
<dd>
<a href="///projects//?fromdocs=">Project Home</a>
</dd>
<dd>
<a href="///builds//?fromdocs=">Builds</a>
</dd>
</dl>
</div>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
Mini Shell