This thesis investigates methods and architectures for machine learning that are particularly suited for the analysis of data emerging from the observation of physical processes and systems. In this context, two main areas have been addressed. The first area is centered around techniques for the detection and characterization of anomalies occurring in a physical process for multi-channel sensor data acquired in real-time. These techniques generalize to the detection of anomalies in multivariate time-series data; secondly, new mathematical approaches, together with a new machine learning architecture, are presented that enable the embedding of a-priori knowledge about the system behavior. This ensures that the solutions obtained are consistent with the physics of the system being observed. This resolves the issue that purely data-driven systems can only make statements about the data; however, to make statements about a system from data, it is necessary to embed a-priori knowledge about the behavior of the system. The second major issue addressed is combining the Rayleigh-Ritz method with machine learning. This required the development of new synthesis algorithms for admissible functions that ensure a hard fulfillment of constraints. The Rayleigh-Ritz Autoencoder embeds the admissible functions in the decoder portion of an autoencoder enabling the hard fulfillment of generalized constraints. This removes the necessity for an additional regularizing term in the cost function. In turn, this permits the learning of the solutions to problems modeled via the calculus of variations, whereby the cost function corresponds directly to the functional that needs to be minimized. Both parts are supported by a hybrid architecture for hyperparameter optimization based on a genetic algorithm. All the above concepts are presented as a collection of peer-reviewed publications and supported by code published in the form of toolboxes.