Reinforcement Learning as a Service

I’ve been integrating reinforcement learning into an actual product for the last 6 months, and therefore I’m developing an appreciation for what are likely to be common problems. In particular, I’m now sold on the idea of reinforcement learning as a service, of which the decision service from MSR-NY is an early example (limited to contextual bandits at the moment, but incorporating key system insights).Service, not algorithm Supervised learning is essentially observational: some data has been collected and subsequently algorithms are run on it. (Online supervised learning doesn’t necessarily work this way, but mostly online techniques have been used for computational reasons after data collection.) In contrast, counterfactual learning is very difficult do to observationally. Diverse fields such as economics, political science, and epidemiology all attempt to make counterfactual conclusions using observational data, essentially because this is the only data…
Original Post: Reinforcement Learning as a Service

Generating Text via Adversarial Training

There was a really cute paper at the GAN workshop this year, Generating Text via Adversarial Training by Zhang, Gan, and Carin. In particular, they make a couple of unusual choices that appear important. (Warning: if you are not familiar with GANs, this post will not make a lot of sense.)They use a convolutional neural network (CNN) as a discriminator, rather than an RNN. In retrospect this seems like a good choice, e.g. Tong Zhang has been crushing it in text classification with CNNs. CNNs are a bit easier to train than RNNs, so the net result is a powerful discriminator with a relatively easy optimization problem associated with it. They use a smooth approximation to the LSTM output in their generator, but actually this kind of trick appears everywhere so isn’t so remarkable in isolation. They use a pure…
Original Post: Generating Text via Adversarial Training

How big compute is powering the deep learning rocket ship

The D.E. Shaw Supercomputer, Anton. (source: Matt Simmons on Flickr).For more on this topic, check out the deep learning session lineup at Strata + Hadoop World San Jose, March 13-16, 2017. Subscribe to the O’Reilly Data Show Podcast to explore the opportunities and techniques driving big data, data science, and AI. Find us on Stitcher, TuneIn, iTunes, SoundCloud, RSS. Specialists describe deep learning as akin to a rocketship that needs a really big engine (a model) and a lot of fuel (the data) in order to go anywhere interesting. To get a better understanding of the issues involved in building compute systems for deep learning, I spoke with one of the foremost experts on this subject: Greg Diamos, senior researcher at Baidu. Diamos has long worked to combine advances in software and hardware to make computers run faster. In recent…
Original Post: How big compute is powering the deep learning rocket ship

The Prior: Fully comprehended last, put first, checked the least?

Priors are important in Bayesian inference. Some would even say : ” In Bayesian inference you can—OK, you must—assign a prior distribution representing the set of values the coefficient [i.e any unknown parameter] can be.” Although priors are put first in most expositions, my sense is that in most applications they are seldom considered first, are checked the least and actually fully comprehended last (or perhaps not fully at all). It reminds of the comical response of someone when asked for difficult directions – “If I wanted to go there, I wouldn’t start out from here.” Perhaps this is less comical – “If I am going to be doing a Bayesian analyses, I do not want to be responsible for getting and checking the prior. Maybe the domain expert should do that or just accept the default priors I find in…
Original Post: The Prior: Fully comprehended last, put first, checked the least?

Custom images for Shiny dashboard valueBox icons

The shinydashboard package provides functions like valueBox that conveniently display basic information like summary statistics. In addition to presenting a value and subtitle on a colored background, an icon may be included as well. However, the icon must come from either the Font Awesome or Glyphicon icon libraries and cannot be image files. I’ve provided a gist that shows how to achieve the use of custom icons with local image files stored in an app’s www/ directory. It involves overriding a couple functions in shiny and shinydashboard and adding a small bit of custom CSS. Ideally, functionality could be included in future versions of these two packages to allow this in a more robust and complete fashion. But for now, here is a way to do it yourself for value boxes. The gist above includes the app.R file to…
Original Post: Custom images for Shiny dashboard valueBox icons

The 2017 machine learning outlook

Tower viewer. (source: Pexels).Join Steven Camiña of MemSQL for “Building the Ideal Stack for Machine Learning,” where he’ll share how to use real-time data for machine learning. Machine learning has been a mainstream commercial field for some time now, but it’s going through an important acceleration. In this podcast episode, I talk about that acceleration with two executives from MemSQL, a company that specializes in in-memory databases: Gary Orenstein, MemSQL chief marketing officer, and Drew Paroski, MemSQL vice president of engineering. Orenstein and Paroski identify a few crucial inflections in the machine learning landscape: machine learning models have become easier to write; computing capacity on the cloud has increased dramatically; and new sources of data—everything from drones to smart-home devices and industrial controllers—have added new richness to machine learning models. Computing capacity and software progress have made it possible to…
Original Post: The 2017 machine learning outlook

Third Actuarial Pricing Game

With the support of ACTINFO Chair and the (French) Institute of Actuaries, our Third Actuarial Pricing Game starts today ! There is a toolbox file available online, with a description of the game : the rules, the dates, and a description of the datasets 3 datasets : one underwriting and one claims databases, for year 0 (training data) and one underwriting dataset to enter the game Anyone can play. Students from various programs around the world, as well as practitioners are welcome to play. It can be by teams, and there are no limit on the size. And there is no registration: to start playing, teams have to submit a dataset before the deadline (end of February), to [email protected]. Related R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data,…
Original Post: Third Actuarial Pricing Game

R/Finance 2017: Call for Papers

R/Finance 2017: Applied Finance with R May 19 and 20, 2017 University of Illinois at Chicago The ninth annual R/Finance conference for applied finance using R will be held on May 19 and 20, 2017 in Chicago, IL, USA at the University of Illinois at Chicago.  The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics.  All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past eight years, R/Finance has included attendees from around the world.  It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2017. We invite you to submit complete papers in pdf format for consideration.  We will also consider one-page abstracts (in txt or pdf…
Original Post: R/Finance 2017: Call for Papers

Fintech in 2017: 4 things to watch

Binoculars. (source: PublicDomainPictures.net).Get the O’Reilly Fintech Newsletter and receive crucial insights and news about the impact of technology on currency, transactions, and capitalism. The following piece was first published in the newsletter. Fintech companies large and small face many of the same disruptive trends as every other kind of tech company—especially the rise of artificial intelligence (AI) and uncertain political outlooks in the United States and Europe. Jon Bruner takes a look at what 2017 might hold in the fintech world. 1. The rise of Chinese fintech China’s eight fintech unicorns—privately held startups valued at more than $1 billion—are together valued at nearly $100 billion, three times as much as America’s 14 fintech unicorns. A couple of them are big because they’re tie-ups between several of China’s largest internet companies, but the breathtaking scale of China’s fintech market is still…
Original Post: Fintech in 2017: 4 things to watch