Making Sense of Big Data

Use Kubernetes to reduce machine learning infrastructure costs and scale resources with ease.

Photo in public domain from wikipedia
Photo in public domain from wikipedia

Kubernetes became the reference for container orchestration. Container orchestration means starting containers, shutting them down, scaling them vertically (quantity of memory and CPU attributed), and scaling them horizontally (number of containers running in parallel).

Does Kubernetes add value to machine learning? Machine Learning needs a lot of resources for training a model, but a little resource for serving prediction. Kubernetes adapts the resources automatically! Machine learning usually implies a long training duration, that can be shortened by distributing the task over several computers. Kubernetes shines in adapting the number of computers necessary. …


A simple solution based on python and celery

Motivation

Let’s consider the following task: predicting the demand for something in several cities. This is a time series forecasting problem. Forecasting differentiates between local models and global models. With local models, you train a model for each city. With global models, you train one model for all the cities. All to say: it’s sometimes necessary to train multiple similar models.

The naive solution consists in training the models sequentially: one model after another. Imagine you need 10 minutes to train the model of a city, and you have 100 cities. You need 1 000 minutes to train all the models…


Comment représenter des formes géographiques ? Le format GeoJSON est un standard conçu pour cela. Il peut représenter aussi des attributs non spatiaux. Cet article présente le GeoJSON, donne des outils pour travailler efficacement avec le GeoJSON et vous présente les pièges à éviter.

Prérequis : être confortable avec le format JSON.

Le point - la base du GeoJSON

+-------------+--------+-------------------------------------------+
| Clé | Type | Valeur |
+-------------+--------+-------------------------------------------+
| type | String | Point, LineString, Polygon, |
| | | MultiPoint, MultiLineString, MultiPolygon |
+-------------+--------+------------------------------------------ + | coordinates | array | Coordonnées [longitude, latitude] |
+-------------+--------+-------------------------------------------+

Exemple

La Feature - enrichir une géométrie d’attributs non spatiaux

Les attributs non-spatiaux sont appelés “properties”.

+------------+--------+------------------------------------+ | Clé…


Le géocodage est une partie importante de la préparation des données spatiale (spatial data preprocessing). Il s’agit d’attribuer une clé à toute paire latitude, longitude.

Geoffroy Gobert

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store