How do I obtain the layer names for use in the iOS sample app? (Tensorflow)

Question

I'm very new to Tensorflow, and I'm trying to train something using the inception v3 network for use in an iPhone app. I managed to export my graph as a protocolbuffer file, manually remove the dropout nodes (correctly, I hope), and have placed that .pb file in my iOS project, but now I am receiving the following error:

Running model failed:Not found: FeedInputs: unable to find feed output input

which seems to indicate that my input_layer_name and output_layer_name variables in the iOS app are misconfigured.

I see in various places that it should be Mul and softmax respectively, for inception v3, but these values don't work for me.

My question is: what is a layer (with regards to this context), and how do I find out what mine are?

This is the exact definition of the model that I trained, but I don't see "Mul" or "softmax" present.

This is what I've been able to learn about layers, but it seems to be a different concept, since "Mul" isn't present in that list.

I'm worried that this might be a duplicate of this question but "layers" aren't explained (are they tensors?) and graph.get_operations() seems to be deprecated, or maybe I'm using it wrong.

Answer 1

As MohamedEzz wrote there are no layers in Tensorflow graphs. There are only operations that can be placed under the same name scope.

Usually operations of a single layer placed under the same scope and applications that aware of name scope concept can display them grouped.

One of such applications is Tensorboard. I believe that using Tensorboard is the easiest way to find node names.

Consider the following example:

import tensorflow as tf
import tensorflow.contrib.slim.nets as nets

input_placeholder = tf.placeholder(tf.float32, shape=(None, 224, 224, 3))

network = nets.inception.inception_v3(input_placeholder)

writer = tf.summary.FileWriter('.', tf.get_default_graph())

writer.close()

It creates placeholder for input data then creates Inception v3 network and saves event data (with graph) in current directory.

Launching Tensorflow in the same directory makes it possible to view graph structure.

tensorboard --logdir .

Tensorboard prints UI url to the console

Starting TensorBoard 41 on port 6006
(You can navigate to http://192.168.128.73:6006)

Below is an image of this graph.

Locate node you are interested in and select it to find its name (in the upper left information pane).

Input: Output:

Please note that usually you need not node names but tensor names. In most cases it is enough to add :0 to node name to get tensor name.

For example to run Inception v3 network created above using names from the graph use the following code (continuation of the above code):

import numpy as np

data = np.random.randn(1, 224, 224, 3) # just random data
session = tf.InteractiveSession()
session.run(tf.global_variables_initializer())
result = session.run('InceptionV3/Predictions/Softmax:0', feed_dict={'Placeholder:0': data})
# result.shape = (1, 1000)

Answer 2

In the core of tensorflow, there are ops (operations) and tensors (n-dimensional arrays). Each op takes tensors and gives back tensors. Layers are just convenience wrappers around a number of ops that represent a neural network layer.

For example a convolution layer is composed of mainly 3 ops :

1. conv2d op : this is what slides a kernel over the input tensor and does element-wise multiplication between the kernel and the underlying input window.
2. bias_add op : adds the biases to the tensor coming out of the conv2d op
3. activation op : applies an activation function element-wise to the output tensor of the bias_add op

To run a tensorflow model, you provide feeds (inputs) and fetches (desired outputs). These are tensors, or tensor names.

From this line of code Inception_model, it seems that what you need is a tensor named 'predictions' which has the n_class output probabilities.

What you observed (softmax) is the type of the op that produced the predictions tensor

As for the input tensor name, the inception_model.py code does not show the input tensor name, since it's an argument to the function. So it depends on what name you have given to that input tensor.

Answer 3

When you create your layers or variable add the parameter called name

with tf.name_scope("output"):
    W2 = tf.Variable(tf.truncated_normal([num_filters, num_classes], stddev=0.1), name="W2")
    b2 = tf.Variable(tf.constant(0.1, shape=[num_classes]), name="b2")
    scores = tf.nn.xw_plus_b(h_pool_flat, W2, b2, name="scores")
    pred_y = tf.nn.softmax(scores,name="pred_y")

In this case I can access final predicted values by using "output/pred_y". If you dont have name_scope, you can just use "pred_y" to get to the values

conv = tf.nn.conv1d(word_embeddedings,
                    W1,
                    stride=stride_size,
                    padding="VALID",
                    name="conv") #will have dimensions [batch_size,out_width,num_filters] out_width is a function of max_words,filter_size and stride_size
# Apply nonlinearity
h = tf.nn.relu(tf.nn.bias_add(conv, b1), name="relu")

I called the layer "conv" and used it in the next layer. Paste your snippet like I have done here