Active learning of regular languages as an approach to neural language models verification.
This work tackles the general problem of verifying the behavior of sequence processing neural networks, specifically neural acceptors and neural language models. The contribution is a framework for extracting formal abstractions of the networks under analysis and verifying whether they satisfy given...
Uloženo v:
| Hlavní autor: | |
|---|---|
| Médium: | doctoralThesis |
| Jazyk: | angličtina |
| Vydáno: |
2024
|
| On-line přístup: | https://hdl.handle.net/20.500.12008/45940 |
| Tagy: |
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| _version_ | 1868890225177001984 |
|---|---|
| author | Mayr, Franz |
| author_browse | Mayr, Franz |
| author_facet | Mayr, Franz |
| author_role | author |
| collection | COLIBRI |
| dc.contributor.none.fl_str_mv | Mayr Franz, Universidad de la República (Uruguay). Facultad de Ingeniería. |
| dc.creator.none.fl_str_mv | Mayr, Franz |
| dc.date.none.fl_str_mv | 2024-09-18T15:58:38Z 2024-09-18T15:58:38Z 2024 |
| dc.format.none.fl_str_mv | 105 p. application/pdf |
| dc.identifier.none.fl_str_mv | Mayr, F. Active learning of regular languages as an approach to neural language models verification [en línea] Tesis de doctorado. Udelar. FI. INCO. PEDECIBA. Área Informática, 2024. https://hdl.handle.net/20.500.12008/45940 |
| dc.language.none.fl_str_mv | en eng |
| dc.publisher.none.fl_str_mv | Udelar. FI. |
| dc.rights.none.fl_str_mv | info:eu-repo/semantics/openAccess Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0) |
| dc.source.none.fl_str_mv | reponame:COLIBRI instname:Universidad de la República instacron:Universidad de la República |
| dc.title.none.fl_str_mv | Active learning of regular languages as an approach to neural language models verification. |
| dc.type.none.fl_str_mv | Tesis de doctorado info:eu-repo/semantics/doctoralThesis info:eu-repo/semantics/acceptedVersion |
| description | This work tackles the general problem of verifying the behavior of sequence processing neural networks, specifically neural acceptors and neural language models. The contribution is a framework for extracting formal abstractions of the networks under analysis and verifying whether they satisfy given requirements. This process steps on two core ideas: 1) treating the neural network as a black box, and 2) using a probabilistic framework to analyze how much the extracted model approximates the original one. For this matter, a series of active learning algorithms and techniques are proposed, developed and analyzed. For the case of neural acceptors a procedure for checking properties of neural networks is presented. This approach is able to check properties without explicitly building representations of the network. We show that this approach offers better guarantees and is more efficient than post-learning verification where the property is checked on a learned model of the network. Besides, it does not require resorting to an external decision procedure for verification nor fixing a specific requirement specification formalism. For the case of neural language models, a learning algorithm based on a congruence over strings which is parameterized by an equivalence relation over probability distributions is presented. The learning algorithm is implemented using a tree data structure and shown to be empirically more efficient than reference techniques. |
| eu_rights_str_mv | openAccess |
| format | doctoralThesis |
| id | anni_ecaf9f82228688eaa498231d3feb0a64 |
| identifier_str_mv | Mayr, F. Active learning of regular languages as an approach to neural language models verification [en línea] Tesis de doctorado. Udelar. FI. INCO. PEDECIBA. Área Informática, 2024. |
| instacron_str | Universidad de la República |
| institution | Universidad de la República |
| instname_str | Universidad de la República |
| language | eng |
| language_invalid_str_mv | en |
| network_acronym_str | anni |
| network_name_str | oai-lr-anni |
| oai_identifier_str | oai:colibri.udelar.edu.uy:20.500.12008/45940 |
| publishDate | 2024 |
| publishDateSort | 2024 |
| publisher.none.fl_str_mv | Udelar. FI. |
| reponame_str | COLIBRI |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0) |
| spelling | Active learning of regular languages as an approach to neural language models verification.Mayr, FranzThis work tackles the general problem of verifying the behavior of sequence processing neural networks, specifically neural acceptors and neural language models. The contribution is a framework for extracting formal abstractions of the networks under analysis and verifying whether they satisfy given requirements. This process steps on two core ideas: 1) treating the neural network as a black box, and 2) using a probabilistic framework to analyze how much the extracted model approximates the original one. For this matter, a series of active learning algorithms and techniques are proposed, developed and analyzed. For the case of neural acceptors a procedure for checking properties of neural networks is presented. This approach is able to check properties without explicitly building representations of the network. We show that this approach offers better guarantees and is more efficient than post-learning verification where the property is checked on a learned model of the network. Besides, it does not require resorting to an external decision procedure for verification nor fixing a specific requirement specification formalism. For the case of neural language models, a learning algorithm based on a congruence over strings which is parameterized by an equivalence relation over probability distributions is presented. The learning algorithm is implemented using a tree data structure and shown to be empirically more efficient than reference techniques.El presente trabajo aborda el problema general de la verificación del comportamiento de redes neuronales que procesan secuencias, en concreto los aceptores neuronales y los modelos neuronales de lenguaje. La tesis desarrolla un marco teórico-práctico para la extracción de abstracciones formales y la verificación de las redes neuronales bajo análisis. Este proceso se basa en dos ideas centrales: 1) tratar la red neuronal como una caja negra, y 2) utilizar un marco probabilístico para analizar en que medida el modelo extraído se aproxima al original. Para ello, se proponen, desarrollan y analizan una serie de algoritmos y técnicas de aprendizaje activo. Para el caso de los aceptores neuronales se presenta un procedimiento de verificación de propiedades de redes neuronales. Este enfoque es capaz de verificar propiedades sin construir explícitamente representaciones de la red. Se demuestra que este enfoque ofrece mejores garantías y es mas eficiente que la verificación posterior al aprendizaje, en la que la propiedad se verifica únicamente sobre el modelo aprendido de la red. Además, no requiere recurrir a un procedimiento de decisión externo para la verificación ni fijar un formalismo específico de especificación de requisitos. Para el caso de los modelos neuronales de lenguaje se presenta un algoritmo de aprendizaje basado en una congruencia sobre secuencias que se parametriza mediante una relación de equivalencia sobre distribuciones de probabilidad. El algoritmo de aprendizaje se implementa utilizando una estructura de datos en árbol y se muestra que es empíricamente mas eficiente que las técnicas de referencia.Udelar. FI.Mayr Franz, Universidad de la República (Uruguay). Facultad de Ingeniería.2024-09-18T15:58:38Z2024-09-18T15:58:38Z2024Tesis de doctoradoinfo:eu-repo/semantics/doctoralThesisinfo:eu-repo/semantics/acceptedVersion105 p.application/pdfMayr, F. Active learning of regular languages as an approach to neural language models verification [en línea] Tesis de doctorado. Udelar. FI. INCO. PEDECIBA. Área Informática, 2024.https://hdl.handle.net/20.500.12008/45940reponame:COLIBRIinstname:Universidad de la Repúblicainstacron:Universidad de la RepúblicaenengLas obras depositadas en el Repositorio se rigen por la Ordenanza de los Derechos de la Propiedad Intelectual de la Universidad de la República.(Res. Nº 91 de C.D.C. de 8/III/1994 – D.O. 7/IV/1994) y por la Ordenanza del Repositorio Abierto de la Universidad de la República (Res. Nº 16 de C.D.C. de 07/10/2014)info:eu-repo/semantics/openAccessLicencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0)oai:colibri.udelar.edu.uy:20.500.12008/459402026-04-14T10:27:58Z |
| spellingShingle | Active learning of regular languages as an approach to neural language models verification. Mayr, Franz |
| status_str | acceptedVersion |
| title | Active learning of regular languages as an approach to neural language models verification. |
| title_full | Active learning of regular languages as an approach to neural language models verification. |
| title_fullStr | Active learning of regular languages as an approach to neural language models verification. |
| title_full_unstemmed | Active learning of regular languages as an approach to neural language models verification. |
| title_short | Active learning of regular languages as an approach to neural language models verification. |
| title_sort | Active learning of regular languages as an approach to neural language models verification. |
| url | https://hdl.handle.net/20.500.12008/45940 |